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ABSTRACT ' ^ 

Item response model applications to National 
Assessment of EducatiQnal Pro^^ data .specif ically aimed at 

the uses of item response models TF mathematics item banking ar^ 
discussed. Approaches for addressing goodness of fit were organized 
into three categories: Checks on model assumptions, expected 
features, and additional model predictions. Within the categories, 
several new methods were also advanced and several plder methods 
which were not in common use for determining item response model-data 
fit were described. Many of these methods were then used to determine 
the fit of the one- and three-parameter models to six NAEP 
mathematics booklets ( three booklets for 9-year-olds and three 
booklets for 13-year-olds) in the 1977-78 assessment. There were some 
inconsistent findings but it did appear that the three-parameter 
model provided an excellent fit to the data sets whereas the 
one-parameter model did not; When a bank of content valid and 
technically Sound test items is available, and goodness of fit 
studies reveal a high match between the chosen item response model 
and the test data, item response models may be useful to NAEP in test 
development, detection of biased items, score reporting, equating 
test forms and levels, item banking, and ot^er applications as well. 
Primary type of information provided by the report: Results 
(Secondary Analysis) . (Author/PN) 
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Abstract 

In view of the technical advances and applications of item response 
models ar;ound the country, it is reasonable to expect ECS to consider the 
potential of item response models for use within its assessment programs. 
Among the areas to which the models could be applied are: 

1. test building (item analysis , item bias, and item selection), 

2. equating test forms, 

■3. measuring .achievement growth, 

4. linking NAEP exercises to other national, state, and district 
tests and test score norms. 

But, the advantages derived from the applications of item response models 
(Stonot be achieved if the fit between the item response model of interest 
and NAEP exercises is less than adequate., Unfortunately , relatively little 
work has been done on the problem of determining the goodness of ^ fit between 
an item re'sponse model and a test data set. Also, no one has looked at the 
fit between any of the item response models and NAEP mathematics exercises. 

The research study had two principal oojectives: 

1. To organize and evaluate many of the available approaches for 
addressing the fit between an item response^model and a data set. 

2. To fit the one- and three-parameter logistic models to several 
NAEP mathematics exercise booklets, and evaluate and compare 
the results . . , ■ * 

Approaches for addressing goodness of fit were organized into three 
categories: Checks on model assumptions, expected features, and additional 
model predictions. Within the categories, several new methods were also 
advanced and several older methods which were not in common use for deter- 
mining item response model-data fit were described. Many of| these methods 
were then used to determine the fit of t,he one- and three-parameter models 
to six NAEP mathematics booklets (three booklets for nine year olds and 
three booklets for thirteen year olds) in the 1977-78 assessment. Th'^re 
were some inconsistent findings but it did appear that the three-parameter 
model provided an excellent fit to the data sets whereas the one-parameter 
model did not. Recommendations for conducting future goodness of fit in- 
vestigations were offered in the final section of the report. 
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1.1 Statement of Problems 

Item response theory, or latent trait theory as it has sometimes 
been called, is the most popular topic for research at the present time 
among measurement specialists^. There are numerous published rese'arch 
studies and conference presentations, and plentiful and diverse applica- 
tions of the theory (for example,, see Hambleton, Swaminathan, Cook, 
Eignor, & Gifford, 1978; Lord, 1980). At least six books on the topic 

are in preparation; Applied Psychological Measurement will devote a special 
* * f . 

issue to the topic in 1982; and the Educational Research Institute of 
British Columbia will pub.lish a special monograph in 1982 on promising 
item response model applications. 

Presently, iteih response theory (IRT) is used by nearly all' of the 
large test publishers , and many , state departments of education and 
industrial and professional organizations to construct tests, to study 
item bias, to equate tests, and to report test score information. The 
many applications appear to be so successful that discussions of IRT have 

3 

shifted from consideration of their advantages and disadvantages compared 
to classical test models to a consideration of topic§ such as model 
selection, item and ability parameter estimation, and methods for deter- 
mining goodness of fit. Nevertheless, it would be incorrect to convey 
the impression that issues and technology associated with it^m response 
theory are fully developed and without controversy. Still, considerable 
progress has been made since the seminal papers by Lord (1952, 1953). 



. In view of the technical advances and appi^ications of item response 
models (IRMs) it is reasonable to expect "that the Educational Commission 
for the States (ECS) will consider in the near future the potential of 
IRMs for use within its assessment programs. Among the areas to which' 
the models could be applied are: o * 

1. in test buildirfgj^ (item analysis, item bias, and item selection); . 

2. in equating t^t frftms;' 

3. in reporting test scores; 

4 . ia, measuring achievement growth (on various groups of feitaminees , 
or on particular test items); 
• 5. in assessing test score reliability?; 

6. in linking NAEP exercises to other national, state, and district 
test score norms.- 

It must be recognized however that any advantages derived from the 
applications of IRMs cannot be achieved if the fit between an IRM and a 
test data set of interest is less than adequate. Unfortunately, to date, 
relatively little research has addressed the problem of determining the 
goodness of fit between an IRM and a test data set of interest. What 
work has be^n done involves statistical tes-ts, but these tests cannot 
be used as the sole determiner of model-data fit because of their 
dependence on examine sample size. When sample sizes are large (a's 
they will be with NAEP test data), nearly all departures between a 'model 
and a data set (even those where th^ practical significance of the 
difference is minimal) will lead to rejection of the null hypothesis of 
model-data fit. With small sample sizes even big differences may not 
be detected via statistical methods because of the low level of statistical 

6 
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power*. ECS and dthers, interested in applying IRMs could benefit from . - 

a set of recommendations for addressing goodness-of-f it studies, 

*> 

Unfortunately, the extant literature has not been compiled or- organized, 
nor, to our knowledge, has much of the literature been critically 
evaluated. «. • " 

Another problem is that the fit between any of the IRMs and NAEP 
mathematics exercises has nqt been studied. Of special interest in this study 
are goodness-of-f it r'esults pertaining to several of the more promising 
applications of IRMs- with the NAEP exercises. One of these applications 
involves creacing an item bank with released items and then "linking" 
all of 'the items at a given age level to a common ability scale, Nbn- 
NAEP items can also be calibrated and added to the bank. In theory, 
statistical descriptors of items ('oltem parameters) that are obtained from 
item response model analyses do not depend upon the choice of examinee 
groups n %^qj i>/n estimating them, and expected ability estimates for. 
examinees' do not depend upon the particular choice of items selected from 
the bank. Such a system would permit, for example, schools to measure 
academic growth even though different test items are used at each test 
administration, • Also,^t would be possible to predict how well groups 
of examinees would have done on selected NAEP mathematics exercises 
(and comparisons can be made to the, reported NAEP item norms) from their 
performance on other test items included in the item bank. Why would 
anyone wish to administer a different set of test items from tho.se items \ 
which were normed? One reason is that teachers may wish to administer 
particular items to examinees because of their diagnostic value, A 
• second reason is that with students who may be expected to do rather poorly 
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or well^ on a test, better estimates of their abilities can be obtained 
when test items are selected to match their expec ted ability level s 
(Hambleton, 1979). There are other uses of item banks as well (see, 
for -'example, Hambleton et al., 1978), Again,' however, these desirable 
outcomes will only be obtained if there is. a more than adequate fit for an 
item response model to the NAEP mathematics exercise data. Of special 
interest is the invariance of item parameter estimates. For example, 
^wheri items ' :^unct ion differently for males and vfemales ; blacks, hispanics, 
and whites; and students from computationally-oriented and non- 
computationally-oriented math programs; IRM assumptions are violated 
and desired outcomes are not achieved. It is important to determine to 
what extent invariance of item parameters is obtained and over which sub- 
populations of examinees because the findings from item invaria?ice studies 
hav^ a direct bearing on the utility of IRMs in item banking. That is, 
when item statistics, are not invariant, the usefulness of t-he item 
statistics, norms, etc., associated with an-item bank are limited. 

In summary, it would appear that there are several reasons for ECS 
to consider t;he utility of IRMs in their'' test development, analysis, and 
score reporting work. However, some preliminary work on approaches for 
assessing goodness of fit must be done fir6t. With the approaches in hand 
a variety of goodness-of-f it studies can be conducted on the NAEP mathe- 
matics exercises. Finally, at this time the advantages and disadvantages 
of the one- and" three-parameter logistic models in relation to the NAEP 
exercises is unknown. Some work in the area would help ECS select the 
proper model, ° if they decide to use IRMs in one or more aspects of their 
testing mettfods and procedures. 
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1,2 Objectives , * \ ' 

The research study had two principar ob j ectives : 

1. To organize and evaluatge many of the- available, approaches for 
addressing the fit between an item response model and a data 
set. ' 

2. To fit the one- and three-parameter logistic models to several 
NAEP mathematics exerciise booklets, and evaluate and compare 
the results . ^ 

The potential of item response theory for solving a ^variety of NAEP k 
testing and measurement problems appears to be substantial. However, this 
promise or potential is not guaranteed .by simply processing test results 
through an available computer program to perform item response model analyses , 
Also, it cannot'' be assumed that because so many other data sets have 
been fit by item response models that the fit to NAEP exercise data is 
assured. The fact is that many of the applications - desc'ribed in the 
literature and especially the large set of AERA, NCME, and NAEP 'confer- 

* .' ^ 

ence papers have failed to adequately address the goodness-'of-f it issue 
and so the extent of model-data fit is unknown. Alsa^ because of the 
nationaI~rmpdft^^^^^ NAEF; it is essential to car'^f ully f 

evaluate any proposed changes or additions to NAEP's approaches for 
building exercises and reporting and using the test information. Pre- 
sently, NAEP is very successful, highly visible, and important. There- • 
fore, there is no reason to take risks in test development and score 
reporting. In this research project* ECS is provided with a framework 
and methods* for addressing the goodneiss-of-fit question. And, the work 
in this area -should impact on other groups who are interested in addressing 
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model-data it question^. Second, ECS is provided witrh information' 
pertaining to the fit between several NAEP inathemaf ics exercise booklets 
and the one- and three-parameter logistic test models. 

1.3 Item Response Models, Assumptions, Busic ConceptsJ ^ 

For many years now the classical test model ha§ been useful to test 
developers and. test score users. The model, is based on^'Wak assumptions"' 
and therefore the model can be applied to many testing probleras (Lord & 
Novick, 1968). But, ^ in spite of the wide acceptance of the classical 
test model, it- has several ifnportant limitations. One limitation is that 
the two most common classical descriptors of test items, item difficulty 
and item discrimination, vary as a function of th.e '^verage ability and the 
range of ability found in the particular sample of examinees for which 
they are ^computed. The usefulness of these, item statistics in building 
tests is limited therefore to groups similar to those from which the 
examinee sample was drawn. Sample-dependent item statist;Lcs are a serious 
handicap for test developers.. 

Another shortcoming is that examinee test scores depen^ upon, the 
particular selection of items included in a test; If distinct samples of 
test i^ems are drawn from a pool of, items all designed to measure the 
same knowledge^and skills , and these item samples differ in difficulty, 
the test scores an examinee can expect to earn on these samples will also 
differ. With item-dependent ability estimates, comparisons among examinees 
are limited to situations where examinees have been administered identical 
(or "parallel") sets of test items. 



^The material in this section of the report was edited from a paper 
■ ' by Hambleton (1979) . 

■ e 
o 



A third shortcoming of the classical test model- is that it assumes 
that the errors of measurem.ent are the same for all examinees. 4&t is not 
uncommon to observe, however, ^that some examinees perform tasks more 
consistently than'others and 'that consistency varies 'with ability. Needed 
are test models which can provide -information abo.ut. the precision of test 
scores and that are free to -vary from one test score to another T 

Because of the shortcomings of the classical test, model, psycho- 
metricians have, been investigating . and developing more appropriate test « 
models. Considerable attention is being directed currently toward the 

' • \ r %^ 

.4 

field of latent trait theory , sometimes referred to as- item response theory 

or item characteristic curve theory (Lord, 1980) ^ , 

In a few words, item response theory po^^lates thavt (1) underlying 
■ / ■ 

examinee performance on a test is a single ability or trait, and (2) the 

relationship^ between examinee' p^rfoimance on each item* and the al5ility 

measured by th^ test can be described by a monotonically increasing curve. 

The curve is called an item charaot^istic oprve* and it provides the 

probability of examinees at various aibility levels answering an item 

correctly. In Figure 1.3.1 below, two item characteristic curves are 

shown. . , * •« 



11 




depends on the level cTf examinee ability. .Examinees with more ability 



have higher probaTDilitt/es for giving correct answers to items than lower 
ability examinees, Item characteristic curves are typically described by 
one-, two— /and three-parameter curve.^ The three item parameters .are, -l 
called item difficulty, item discrimination, and item, pseu 'o-chance level. 
Items which are shifted to the right end of the ability scale are more 
difficult than those shifted *£o the left end of the ability scale. It is 
clear from Figure 1.3.1 then .that, item 2 is more riifficuit than item 1. 
''The slope of an item characteristic curve describes an item's discriminat- 
ing power. In Figure 1.3.1, thejtefore, item 2 is more discriminating 
than item 1. Finally, the probability of a very low ability ,examijiee 
answering an item correctly .is the item's pseudo-chance level. With 
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item 1 in the figure, the probability is^v-.^With item 2 the probability 
is somewhat higher. j 

Most item response models, and all of the models which are presently^ 
popular, require the assumption that the test items are homogeneous in ^ ^ 
the sense that they measure a single ability or trait. ^ In addition, it 
is common to assume that the item, characterfsti^c curves are described 

by one-, two-, or three-parameters, and the corre'sponding models are 

\ 

referred to as one-, two-, and three parameter models, respectively. 
With the three-parameter model, items ckn vary in their diff'iculty, 
discrimination*^ level, and pseudo-chance level. With the two-parameter 
model, the pseudo-chance level parameter is 0 for all items. ^ With the 
one-parameter model, not only does ^the pseudo-chance level parameter have 
a value of 0 for all items', but all items have a common level of dis- 
^crimination, 

Wlien' the assumptions of item -response theory can be met in the ■ 
data s,ets to which it is applied,, at least a reasonable degree, what 
is obtained arfe. (1) examinee ability estimates in the pool of items from 
which the items are drawn that do not depend upon the particular sample 
of items selected for the test, (2) item descrip-tors or statistics (diffi- 
culty ,, discrimination, pseudo-chance level) that do not depend upon the 
particular samp4.e of examinees from the pcfpulation of examinees for whom 
the earlier mentioned item pool is suitable, and (3) a statistic is pro- 
vided Indicating the precision with which each examinee!? ability is es- 
timated. Of course; the extent to which the three advantages are gained 
in an application of an it^em response model depends upon, the closeness 
of- Che **fit** between a se^t of data and the mo.del. If the fit is poor. 



-10- 

the three desirable features either will not be obtained or obtained in 
a low degree (Lor'H, 1980). 

Item response model's are based on a 'set of assumptions about the test 
data. Two of these assumptions will be discussed here: Dimensionality, 
and the mathematical form chosen for the item characteristic curves. 
With respect to dimensipnality , it is common to assume that only one 
ability is necessary to "explain," or "accoUftt" for examinee test per- 
formance. Item response models in which a single latent ability is pre- 
sumed sufficient to explain or account for examinee performance are 
referred to as unidimens ional models. The as^sumption that a set of test 
items is unidimensional is commonly made because scores on tests that 
measure only one trait are relatively easy to interpret. There exists 
no well accepted method for studying the unidimensionality of a set of 
test items. Factor analysis is the most coramon of the psychometric ap- 
proaches used to address the dimensionality quest^ion (Hambleton et^al., 
1978). 

An item characteristic curve (ICC) is a mathematical function that 
relates examinee probability of success on an item to the ability measured 
by the set of items contained in the test. Pj_(6) designates the prob- 
ability of a .correct response to item i by an examinee with ability level 0 
The main difference to be found among currently popular latent trait models 
is in the mathematical form of the ICCs. 

Birnbaum (1968) proposed ICCs which take the form of two-parameter 
logistic functions: 

Dai(e-bi) 

^i^^> - ,te^a,(e-b,) ^ 2, n). . [1.3.1] 

> 
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In equation [1.3.1], Pj^CS) is the" pTo1>ability that an examinee with ability 0 
answers item i correctly, and b^^ and a^^are parameters of item i. The 
parameter, b^, is referred to as item diff-iculty . It is the point on the 
ability scale such that examinees who possess that amount of ability have 
a 50% chance of answering an item correctly. The parameter, a^, called 
item discrimination , "is prop.oj'.tional to the slope of P-lC0) at the point . 
0 = b^. The constant D is- a scaling factor set equal to 1.7. ^ 

A three-parameter model can be constructed from the two-parameter 
model by adding a third parameter, denoted c^. The form of the three- 
parameter logistic curye is 

^Dai(0-bi: 
iW 



P.(0) = c, ^ (l-c,V ^"^^3,(0-b,) ' ^^^^^ 2, n). [1.3.2] 



The parameter is the lower asymptote of the ICC and gives the probability 
of low ability examinees correctly answering the item. 

The one-parameter model (sometimes called the "Rasch model") is a 
special case of the three-paramefter logistic model in whicK guessing 
behavior is minimal (c-j^=0), all items are assumed to have equal discrimin- 
ating power, and items vary only in terms of difficulty. Therefore, 
• 0-bi ' 
1+e 1 

The scale on which ability estimates are located is arbitrary. The . 
scale is chosen so that ICCs of the form specified by the model under 
investigation fit as closely as possible to the a^vailable test data. An 
assumption is made that the correct metric is the one which maximizes 
predictions between unobservable characteristics (ability and item 
parameters) and observable data (examinee item responses). Since both -a 
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.intern difficulty indices and ability estimates are measured on a common 
scale, it is usual to set the mean and standard deviation of one of the 
two variables to 0 and 1, respectively. In fact, any linear transform- 
ation of ability scale units is permissible and predictions from the mode 
will not be influenced so long as the item discrimination parameters are 
revised accordingly. This means, for example, that if an agency wanted 
scores from an instrument on a scale with mean ability = 100 and standard 
deviation = 10, then, ability scores and item difficulties must be trans- 
formed using the linear equations . ^ 

Q^' = 10 + 100 

a ^ 
b* = 10 b. + 100 , 

and che values of a. must be transformed by the equation 




This last equation is determined so that 

4 K - = - bi) . 

If this were not the case, the predictions for the model would be in- 
fluenced by a change in the ability scale (Hambleton, 1980) . 
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2,0 Goodness of Fit Approaches ^ 

'2 , 1 Overview 

Item response models offer a number of advantages for test score 
interpretations and reporting of NAEP results but the advani:ages will 
only be obtained in practice when there is a close match between the 
model selected for use and the test" data. 

From a review of the relevant literature it appears that the deter- 
nnlTi3'tlT3Ti-~-0"f^how-wett--a~-model-accoun^ — — — ~. 



addressed in at least three ways: i 

! 

a. Determine if the test data satisfy the assumptions ; of the 
test model of interest. 

b. Determine if the e^xpected advantages deprived from the use of 
the item response model (for example, invariant item and 
ability estimates) are obtained. \ 

c. Determine the closeness of the fit between predictions and* 
observable outcomes (for example, test score distributions) 
utilizing model parameter estimates and the test d^ta. 

Strictly speaking, tests of model assumptions are not tests of goodness of 
fit but because of their centrol role in model selection and use in the 
interpretation of goodness of fit tests we have included them first in 
a series of desirable goodness of fit investigations. 

Promising practical approaches for addressing each category above 
will be addressed in subsequent sections. First, however, the inappro- 
priateness of placing substantial emphasis on i^esults from statistical 
tests will be explained. 



^Small sections of this chapter are f rom..Ha.mbletoT) et al (1978) and 
Hambleton (1980). . i 
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2.2 Statistical Tests of Significance 

Statistical tests of goodness of fit of various item response models 
hnve been ftTven by many authors (Andersen, 1973; Bock, 1972; Mead, 1976; 
Wright, 'Mead, & Draba, 1976; Wright & Panchapakesan , 1969; Wright Si Stone, 
1979). The procedure advocated by Wright and Panchapakesan (1969) for 
~> testing the^ fit of the one-parameter model is one of the most commonly 

used. It essentially 'involves examining the quantity f.. where f . . repre- 
sents the frequency of examinees at the ith ability \^evel answering the 
j th item correctly. Then,. the quantity V^y where 

is distributed pormally with zero mean and unit variance .^^^ Since f^^ has 

a binomial distribution with parameter P^y probability of a correct 

response is given by e*/(et + b*) for the one-parameter model, and r^, 

the number of examinees in the score group. Hence, E(f_j^j) = '^^P^y 

Var.(f .) = r p .(1-p,.). Thus, a measure of the goodness of fit, 
iJ 1 ij ij ^ 

the model can be defined as 

n-1 n , 

^ 'Z y^. 
i=l i^l-' 

The quantity, defined above has been assumed by Wright and his 

colleagues to have a distribution with degrees of freedom (n-1) (n-2) 
since the total number of observarions in the-matrix ^"^^j^j^ is h(n-l) , 
and the number of parameters estimated is 2(n-l). Wright and 
Panchapakesan (1969) also defined a goodness-*of-f it meas^ure for. 
individual items as ' 
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^ 1=1 



where \(? is assumed to be distributed as with degrees of freedom, (n-2) , 
This me'thod for determining the goodness of fit can also be extended^to 
the two-\ and three-parameter item response models although it has not been 
extended to date- 

There\ are ^several problems associated with the chi-square tests of 
fit discussW above "'The test has dubi6us validity when any one of 
the E(f . ,)'* terms , ^ = 1, 2, , , . , t]^ - 1 ; = 1 , 2, . . . , n , have values less 
than one. This follows from the fact that when any of the E(f^j). terms 
are less than one, the deviates V^y 1-1>^> •••> n-1; j = 1, 2, 
n, -ate— no-t- normally distributed and a x^ distribution is obtained only by 
summing the squares' of normal deviates. Another problem encountered in 
•using the x^ test is that it is sensitive to Sample size. If enough 
observations are tak^n, the null hypothesis that the model fits the data 
will always be rejected usirig^Ke^^ te'st, Eiivgi (1981) and Wollenberg- 
(1980, 1982a, 1982b) have also demdostrated that the Wright-Panchapakesan 
goodness-of-f it statistic is not distributed as a x^ variable and the 
associated degrees of freedom have been assumed to be higher than they 

actually are. Clearly ther^ are substantial • reasons for not relying on 

\ • \ 

the Wright-Panchapakesan stat^istic because of tt>^ role sample ^size plays 

in its interpretation and because of questions c'bnci^rning the appropriate 
sampling distribution and degrees of freedom. 

Alternately, Wright, Mead, and Draba (1976) and Mead (1976) have 
suggested a method of test of fit for the' one--parameter model which in- 
volves conducting an analysis of variance on the variation remaining in 
the data after removing the effect of the fitted .model, ^ This procedure 
allowk not only a determination o.f the general fit of the data to the 
modeL but\also enables the investigator to pin-point guessing as the major 



factor contributing to ti.e misfit. This procedure for testing goodness 

of fit of the one parameter model involves computing residuals in the 

-data after removing the effect of the fitted models These residuals are 

plotted against (0.-b ) .• According to the model, the plot should be ^ 

i g • ■ 

■represented by a horizontal .line through the origin. For guessing, the 
residuals follow the horizontal line until the guessing becomes important- 
Wlien this happens the residuals are positive since persons are doing better 
than expected and in that region, have a negative trend. If practice or 

"^p^a^ Ts' iivvolWd~th'^^ 
with a negative trend line over the entire range of ability. Bias for "a 
particular group may be detected by plotting the residuals separately for 
the two^grcrups. It is generally found that the residuals have a negative 
trend f or' the unf avored-^group and a positive trend for the favored group. 

.Wien maximum likelihood estimates^o-f-^^the^parameters are obtained, 
likelihood ratio tests can be obtained for hypotheses oT^rnterest (Waller, 
1981) - Likelihood ratio tests involve evaluating the ratio ^ X, of the 
maximum values of the likelihood function under the hypothesis of interest 
to the maximum value of the likelihood function under the' alternate 
hypothesis. If the number of observations is large^ -2 log. A is known 
to have a chi-square distribution with degrees of freedom given by the 
difference in the nurrber of parameters estimated under the alternate 
and null hypotheses. An advantage possessed by likelihood ratio tests 
over the other tests discussed earlier is apparent. Employing the like- 
lihood ratio "criterion, it is possible to assess the fit of a, particular 
latent trait model against an alternative. 

o ...... • • 



Anderson (1973) and Bock and Liebermann (1970) have obtained 
likelihood ratio tests for assessing the fit of the Rasc]^ model and 
the two-parameter normal ogive model respectively, Andersen (1973) 
obtains a conditional likelihood ratio test for the Rasch model based 
on the within score group estimates and the overall estimates of item 
difficulties. He shows further that -2 times the logarithm of this 
ratio is distributed as with degrees of freedoin, (n-l) (n-2). 
Based on the work of Bock and Liebermann (1970) , likelihood ratio 
tests can be obtained for testing the fit, of the two-parameter noYmal 
ogive model. It should be pointed out that these authors have 
obtained 'both conditional and unconditional estimates of the para- 
meters. For the likelihood ratio test, it would be more appropriate 
if the unconditional model is used since with this model ability 
parameters are not estimated, and hence, the likelihood ratio criterian 
can be expected to have the chi-square distribution. This procedure 
can be extended to compare the fits of one model against another 
(Andersen, 1973) . 

The major problem with this approach is that the test criteria 
are distributed as chi-square only asymptotically. But, as was . 
mentioned earlier, when large samples are used to accommodate this fact 
the chi-square value may become significant owing to the large sample 
^^^sixe! 



3^.3 Checking Model Assumptions 

Item response models are based on " strong assumptions which will not 
be completely met by any set of test data (Lord &..Novick, 1968). There 
is evidence ,that the models are rbbust to some departures but the 
extent of robustness of th,e' model's has not been firmly established 
(Hambleton et al. , 1978) . Given doubts of the robuatne.^ss of the models, 
one might be tempted to simply fit the most general model since it will 
be based on the least' restrictive assumptions. Unfortunately, the more 

. ■ . , * . _ ; , _ ^ _ ,., , . 

general models^ a re muTt i- imen s ional (i.e., assume that more than one 
latent variable is required to account for examinee test performance), 
.and they are complex and do nofappear ready for wide-scale use. Alter- 

* o *» 

nately. It has been suggested that the three-parameter logistic model, the 
most general of the ♦ uhidimensional models ''in common use, be adopted. In 
theory," the three-parameter model should result in better fits than • 
either the one- or two-parameter models. But, there are three problems 
vjith this course of action: (1) more computer-time is required to con- 
duct the analyses, (2) somewhat larger samples of examinees and items 
are required to obtain satisfactory item and ability estimates, and ^ 
(.3) the additional item parameters (item discrimination and pseudo-chance 
levels) complicate the use of the model for practitioners. oi course, 
in spite of the^roblems, and with important testing programs such as 
NAEP and a- highly trained staff, the three-parameter .model may still be- 
preferred. ^ . ' 

•Model selection can be aided by an investigation of four principal 
assumptions af several of the item response models: unidimensionality , 
equal di scrimination • indices , minimal guessing, and non-speeded test 
administrations. Promising approaching ;^f or studying these assumptions 
are summarized in Figure 2.3-i and will be briefly considered n^.xt. 

.If 
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Figure 2.3.1 Approaches for Conducting Goodness o'f~Fi-t— I-n-vestlgat\ons 



\ " Checking Model Assumptions 

1. ^Unidimensionality (Applies to Nearly AIL Item Response Models) 

• Kuder-Richardson Formula 20 -(Common Approach But Not Acceptable^- 
Stati^tic is Influenced By Test Score Variability and Test Length). 

• IMot of Eigenviilues (From Largest to Smallest) of the Inter-Item 
Correlation Matrix - Look for a Dominant First Factor, arfd a High 

; Ratio of the First to ,,the Second Eigenvalue (Reckase , 1979). 

• Comparison of Two Plots of Eigenvalues -s"^ the One Described Above 



Random Data (Same Sample Size, and_ Number of Variables, Random Data 
Normally Distributed) (Horn , - PsycHometrika , X965) . . 

• Plot of 'Content-Based Versus Total-Test Based Item Parameter Estimates 
(Bejar, JEM, 1980) . ' 

• Analysis of Residuals After Fitting a One Factor Model to the 
.Inter-Item Covariance Matrix (McDonald, B JMSP , 1980). 

Equal Discrimination Indices (Applies to the One-Parameter Logistic 



Model) 



4 



• Analysis of Variability of Item-Test Score Correlations (For Example, 
Point-Blserial and Biserial Correlations). 

• Identification of Percent of Item-Test Score Correlations Falling 
Outside Some Acceptable Range (For Example, the Average Item-Test 
Score Correlation t .15). ^ 

3. Minimal Guessing (Applies to the One- and Two-Parameter Logistic Mad.el) 

• Investigation of Item-Test> Score Plots (Baker, JEM,' 1964, 1965). 

• Consideration of the Performance of Low-Ability • Examinees (Selected, 
with the Use of Test Results, '^r Instructor Judgments) on the Most 
Difficult Test Items. • * ' 



A. 



• Consideration of Item Forftat and Test Time Limits (For Example, 
'-Consider the Number of Item Dis-tractors , and Whether or Not the Test 

Was Speeded) . 

Non-speeded (Power)' Test Administration (Applies to Nearly All Item 
Response Models). ^ ' 

• Comparison of. Variance o'f the Number of Items (Jnattempted to .the 
Variance of the Number of Items Answered Wrongly .(Gulliksen, 1950) .' 

• Investigation of the Relationship Between Scores on a Test With the 
Specified Time Limit and With an Unlimited Time Limit (Cronbach and 
Warrington, 1951) . 



^ _ _ 

• Investigation of (A) Percent of Examinees "Completing the Test, (B) 
Percent of Examinees Completing 75% of the Test, and (C) Number; of 
Items Completed by 80% of the Examinees pTS Method, See Donlon, 1978). 

sChecking Expected Model Features 

Ir Invariance- of Item Parameter Estimates (Applies to All Models) . 

• Comparison of Item- Parameter Estimates Obtained in Two or More" ^ 
Sub-groups of the P pulation for Whom the Test is Intended (For 
Exainple, Males and Females; Blacks, Whites, and Hispanics; 
Instructional Groups; High and Low Perfcrmers on the Test or Other 
Criterion Measurer-Geographic Regions), Normally Comparisons Are 

..„.„._™J.lade_a£..„LkeJLteTnJa 

Form (Scattergrarns) . Random Splits of the Population Into Sub-groups 
**The Same Size Provide a Basis for Obtaining Plots Which Can Serve 
as a Baseline for Interpreting thie Plots of Principal Interest. 
Graphical Displays of Distributions of Standardized Differences in Item 
Parameter Estimates Can Be Studied. Distributions Ought to Have a 
Mean of/Zero and a Standard Deviation of One (For Example, Wright, 
1968; Lord, 1980; Hambleton and Swaminathan,. 1982) . 

2. Invariance of Ability Parameter Estimates (A^^plies to All Models) 

• Comparison of Ability Estimates Obtained in Two' or More" Item Samples 
From the Item Pool of Interest.. Chodse Itein Samples T^^hich Have 
Special Significance Such As Relatively Hard Versus Relatively Easy 
Samples, and Subsets Reflecting Different Content Categories Within 

the Total Item Pool. Again, Graphical Displays and Investigation - ' 

of tl:ie Distribution of Ability Differences Are Revealing. 

Checkiag Model Predictions of Actual (and Simulated) Test Results 

• Investigation of Residuals and Standardized Residuals of Model-Test Data 
Fits at the Item and Person Levels. Various Statistics are Available 

to Summarize the Fit Information. Graphical Displays of Data Can Be 
Revealing. 

• Comparison of Item Characteristic Curves Estimated in Substantially 
Different Ways (For Example, Lord, Psychometrika , 1970). 

• Plot of Test Scores and Ability Estimates (Lord, Psychometrika , 1974). 

• Plots of True and Estimated Item and Ability Parameters (For Example, 
Swaminathan, 1981; Hambleton and Cook, 1982). These Studies Are 
Carried Out With Computer Simulation Methods. • 

• Comparison of Observed and Predicted Score Distributions. Various 
•Statistics (Chi-Square , For Example) and Graphical Methods Can Be 

Used to Report Results. Cross-Validation Procedures Should Be Used, 
Especially If Sample Sizes Are Small (Hambleton and.Traub, BJMSP, 1973). 

• Investigation of Hypotheses Concerning Practice Effects, Test Speeded- 
ness. Cheating, Boredom, Item Format Ef fects , . Item Order, etc. 



Unidimensionality 

The assumption of a unidimension^'l latent: space is a comnipn one for 
test constructors, since they usually desire to construct urildimensional 
tests .so as to enhance the int'erpretability of ^ set ol " test scores 
(Lumsden, 1976). IVJiat does it mean to say tliat a test is unidimenrjlonal? 
Suppose a test consisting of n items is intended for use in r subpopula- 
tions of examinees (e.g., several ethnic groups). Consider next the- 

13 

conditional distributions of test scores at a particular ability level 
for the r subpopulations • These conditional distributions for the r 
subpopulations will be identical if 'the test, is unidimensional.' If 
the conditional distributions vary across the r subpopulations, it' 
can only be because the test is measuring something "other than the •/ 
single ability. Hence,., the test cannot be unidirr\ensional . 

It is possible for a test to be unidimcnsional V7ithin one popula- 
tion of examinees and not unidimensional in another • Consider a test 
with a heavy cultural loading. This test could appear to be unidimensional 
for all populations v/ith the same cultural background. However, when 
administered to populations with varied cultural backgrounds, it may in - 
fact have more than a single dimension underlying the test score* 
. Examples of this situation are seen when the factor structure of a 
particular set of test itcnrs varies from one cultural group to another. 

Lumsden (1961) provided an excellent review of methods for con- 
structing unidimensional tests. He concluded that the method of factor* 
analysis held the most ^ promise. Fifteen years later he Reaffirmed his 
conviction (Luinsdcn, 1976). Essentially, Lumsden recommends that a 

r 
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test constructor generate an initial t6q'o1 of test itcm.s selected on the 
basis of empirical evidence and a priJori grounds. Such an item selection 
procedure will inciease the likelihoid tliat ' a unidimcnsional set of ^ 
test items, within the. pool of items can be found. If test items are not 
preselected, the pool may be too heterogeneous for tlie unidimensional 
-set of items in the iLcin pool to emerge.. In Lumsden's method, a factor 
analysis is performed and items not measuring the dominant factor ob~ , 

-t-ained-Hin -the -f-ae toi^ - -solu tion- are- t emoyed v The^ r ema ining Irems— a r e 

• ^ \ ■ ■ ■ , • ^ 

factor analyzed, and again, "deviant" items are ^removed. The process is 

repeated until a satisfactory solution, is obtained. Convergence i^s most 

likely when the initial item- pool is carefully selected to include only 

items that appear- to be mea&uring a conimon t}~ait. Lumsden proposed that 

the ratio of first factor variance to second factor variance. be used 

aSi an "index of unidimensionality , " 

Factor analysis can als,o be used to check "the,- reasonableness of * 

the assumption of unidimensionality with a set* 0& test items (Hambleton 

& Traub, 1973). However, the approach «d.s not. without problems. F^pr 

exampl-c\ much has been written about the merits of using tetrachomc 

* A ■ 

correlations or phi correlations (McDonald & Ahlawat> 1974). The common" 

• V- ■ ' ■ ■ ' ^ ■ 

beliCif is tWat using phi correlations will tead to" a factor solution 
with too many factors, some of them "difficulty factors'* found because 
of the range of item difficulties among the items in the pool, * McDonald 
and Ahlawat (1974) concluded that "difficulty factors" are unlikely' ijC 
the range oC item difficulties is not extreme and the items are not coo 
higlily discrinunati ng*. ■ - . ^* 
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Tetrachoric correlations have one attractive feature. A sufficient 

condition for the unidimensionality of a set of items is that the matrix 

of tetracliorU^ 1 tom' inrorrorrc^J-ntions has only one common factor (Lord 

{ ■ . . , 

& Novick, 1968). On the nega'fei.ve side, the condition is not necessary. 

Tetrachoric correlations are awkward to calculate (the formula is complex 

and requires some numerical integration), and,. in addition, do not 

necessarily yield a correlation matrix that is positive definite, -a ^ 

problem when factor anllysfs is attempted. ■ " • 

Kuder-Richardson Formula 20 has on occasion " been recommended and/or 
used to address the dimensionality of a set of test items., But Green^' Lissitz 
and Mulaik (1977) have noted that the value of .KR-20 depends on test length and 
group heterogeneity and therefore the statistic provides misleading info?rmation 
about ^unidimensionality . x ' , 

A somewhat more promising method involves considering the plots of 

eigenvalues for test item intercorrelatiou^ ma'trices and looking for the 

d- ^ - ' \ ^ 

**breaks" in the plots to' determine the number .of "significant** underlying 

factors. To assist in locating a "break'* llbrn (1965) -sifggested that the 

plot of interest b'e compared to a plot 'of eigenvalues obtaining from an 

item intercorrelation matrix of the same size and where inter-item » 

correlations are obtained by^ generating random variable's from normal 

distributions. The same number of examinees as used- in t,he correlation 

matrix of interest is simulated. ,* ■ ' . . , 

Another promising ^approach,^ in part because it is- not based on the 

Analysis of correlation coef f icients ,V was suggested by Bejar (1980)': 

1, Split test items on an a priori, basis (i.e., content considerations), 
Por example, isolate a subset of test items which appear to be 
tapping a different ability from the remaining test items.. 

I • o ' 

2, ' For items in the subset, obtain item parameter estimates twice: 

once by including the test items in item calibration for the 
total test and a second time by calibrating only the items in 
^ the subset, ''O^V * - ' 



3.;* Compare the two sets of item parameter estimates by preparing 
/ ->a%plot (see Figure 2,3.2). - 

' ' V i 

Unless ^th^ item parameter estimates (apart from sampling error) are equal, 
the p^robability for passing items for fixed ability levels will differ,' 
This is not acceptable because it implies that performance on items 
depends on which items are included in the test which contradicts the 
unidimensionality assumption. 

Finally, McDonald (1980a, 1980b) and Hattie (1981) have suggested 
trie use of non-linear- factor analysis and the analysis of residuals as 

o 

a promising approach. The approach seems promising because test items 
are related to one ajiother in a non-linear way anyway, and the analysis 
of residuals, after fitting a one-factor solution seems substantially more 
revealing and insightful than Conducting significance tests on the amount 
of variance accounted for, 
\ ■ " 

Equal Discrimination Indices . o 

This assumption isVmade with the one-parameter model. There appear 
to be only descriptive methods avallaSle for investigating departures 
from,, this mod^l assumption, A rough check of its viability is accomp- 
lished by comparing the similarity of item point-biserial or« biserial ' 
correlations. The range (or the standard deviation) of the discriminatdon 
in*dices should .be small if the assumption is to He viable, Wright and 
Ills colleagues have, on occasion, looked at the residuals remaining 
after fitting a one-parameter model and attempted to study variation 
in item discrimination indices but they have written little on their 
methods , 
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Figure 2.3.2 Plot of content-based and total-test based- item 
parameter estimates. . ■ 
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Guessing 

There appears to be no direct way to determine if examinees guess the 
answers to items in a test. Two methods have been considered' (1) non-linear 
item-test score regression lines, and (2) the performance of low test score 
examinees on the hardest test items. With respect to the first method, for 
each test item, the proportion of correct answers for each test score group 
(small test score groups can be combined to improve the accuracy of results) 
are plotted. Guessing is assumed to be operating when test performance for 
the low performing 'score, groups exceeds zero. For method two, the performance^ 
of the low-Scoring examinees on the hardest test questions if of central concern. 
"Neither method however is without faults. The results will be misleading if the 
test items are relatively easy for the low ability group, and/br if the low 
ability group is only -relatively low in ability in relation to other examinees 
in the population of examinees for whom the test is intended but now low ability 
in |an^ absolute sense (i.e., very low scorers on the test). 

Speededness of the Test 

Little attention is given to this' seldom stated assumption of many item 
respjonse. models. When it operates it introduces an additional factor, in- 
J luejncin^ test performance. It can be identified by a factor analytic study. 
Interestiyigly-,, with some of the new ability estimation methods (Lord, 1980), 
the failure of examinees to complete a test can be' handled so that the speed- 
• idnpiss ' fadtor does not /'contaminate" 'ability score estimates. The appropriate- 
fiess of the assumption in- relation to a set of test results can be checked by 



de 



ter 



mining the number of examinees who fail to finish a test- and the number of" 



items they fail to complete. The ideal situation occurs when examinees have 

suffilcient time to attempt each question in a test. • 

\ 

Donlon 1(1978) provided an extensive review of methods for determining 

\ ■ \' ' ... 

the speededness of tests. Three of the most promising are cited in 
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Figure 2.3M, Perhaps discussion of only one here will suffice. It involves 
obtaining an estimate of the correlation- between s,cores obtained under 

power and speed conditions and correcting the correlation for attenuation 

' " 1 . 

due to -the unreliability associated with the' power and speed scores: 



p(Tp,T^) = 



/p(Xp,X^) /p(x^,x;) 



s '"s^ 



The speededness index proposed by Cronbach and .Warrington (1951) is 

Speededness Index = 1 - p^(T^,T ) . . ' ! 

P I 

■The index is obtained in practice by administered parallel-forms of the 

test of interest under speed and power conditions to the samh group of examinees 



2. A Checking Model Features 

When item response models fit test^ data sets, three ^adv^ntages are 
obtained : 



1. Examinee ability estimates are. obtained on the same 
scale and can be compared even though examinees may 
taken different sets of test items from the pool of 
items measuring the ability of ^interest. 



ability. 

have 

test 



2. Item statistics are obtained which do not depend ^ oh j the 

sample of examinees used in the ^calibration of test litems. 

3'. An indication of the precision of ability estimates ;at each 
point on the ability scale is obtained. | " 

It 'is to obtain the advantages that item response models are jof ten chosen 

.a^ the mode of Analysis, However, whether or not these features are- 

■I . . • . . " ' • ... I 

obtained in any application depends on many factors — model-data fit, 

#• ■ ■ •■ ... 

test length, precision of ..the item parameter ^estimates , and slo on, 

! ' . . 

Through some fairly straightforward methods, these features dan be studied 
and their presence in a given , situation determined. j ^ 



The first one can be addressed, for example, by adminis tering examinees two 
more samples of tes't items which vary widely in difficulty (Wrig|||, 1968), 
In some instances, items can be administered in a single test and two 
scores for each examinee obtained: the scores are based on the easier 
and harder -halves of the test. To determine if there is substantial 
difference in test dffficulty, the distributions of scores on the two 

i> ' ■ ■ 

halves of ,the test can be compared. Pairs of ability estimates obtained 
from the two halves of the test for each examinee are plotted on a graph. 
The .bivariate 'plot of ability estimates should be linear because expected 
ability scores for examinees do not depend upon the choice, of test items' 
when the item response model under investigatior\^ fits the test data. 
Some scatter of points about a best fitting line, however, is to be ex- 
pected because of measurement error,"' When a linear relationship is not ^ 
obtained, one or more of the underlying "assumptions of the ■ item response 
model under investigation are being violated, by the test data set. Factors 
such as test characteristics, test lengths, precision of item statistics, 
and so on can also be studied to determine their influence. 

The second feature is studied in essentially the same way as the 
first. The difference is that extreme ability groups. are formed and item 
parameter estimates in the two samples are compared, Wright (1968) and 
Lord (1980) have carried out extensive studies in this area, Again^ if 
the test .data are fit by the item response ^model under investigation, 
there should be. a linear relationship between item parametesr estimates , • 

. ' ^ ■ ' . • . \ . . ' . ; * ^ • . ' . 

from tiie' two examinee samples, even if the samp-l'es differ in ability^ • ' 
race, or sex (Lord & Novick, 1968), The comparison is carfied out for 
each item parameter in the model of interest. With respect 'to NAEP 
exercises it seems especially important 'to compare item parameter estimates 



derived from (say) black and white examinee groups. This check would he- 
a- stiff one but a linear relationship must still be obtained or it must 
be said that the item response model does not fit the test, data for one 
or two of the groups. , . ■ 

Perhaps the most serious weakness of the approaches described above 
(and these are the only ones found in the literature) is that there is no 
baseline data available for interpreting the plots. How is one to .know 
whether the amount of scatter is appropriate, assuming inodel-data fit? 
Alternately, statistical tests are performed to study the differences 
between (say) b values obtained in two samples. But-, as long as,, there 
is at. least, .a small difference in the true parameter values in the 
samples, statistically significant differences will be obtained when 
sample sizes are large; Thus, statistically significant differences may 
be observed even when the practical differences are very small. 

"The third feature is a harder one to address. Perhaps it is best 
answered via simulation methods. According to the theory, if h test is 
"long enough," the conditional distribution-of^a:biTity estimates at 

w ■ _^ 

each ability level is normal (mean = ability; sd = l//inf ormation) . It 
appears that a test must include about 20 items (Samejima, 1977). 

2.5. Checking Additional Model Predictions 

Several approaches for checking model predictions were introduced 
in Figure .2.3il, One of, the most promising approaches for addressing t 
model-ciata fit' inv6 Ives the use of residual analyses. An item resporiJBe 
model is chosen; item and ability parameter estima'tes are^obtained ; and 
predictions of^ the performance of various ability groups on the items on 
the test are made, assuming the validity "of the chosen model. Com- 
parisons of the predicted results with the actual results are made. 



By comparing the average item performance levels of -various ability 
groups to the performance levels predicted by an estimated item- characteristic 
curve, a measure of the fit between the estimated item characteristic 
^^urve and the observed data can be obtained. This process, of course, 
can and is repeated for each item in a .test. In Figure 2.5.1, a plot 
of the residuals (difference between the observed data and an estimated 
item Characteristic curve) across ability groups for four items are reported 
along with likely explanation^ for the results. The average item per- 
formance of each ability group is represented by the symbol *'x'' in the 

' figure. If, for example, 25 of 75 examinees in the lowest ability group 
answered arf item correctly, an '"x" would-be placed at a height of .33 above 
the average ability score in the ability group where the performance was 
obtained. (The width of each ability group should be wide enough to con- 
tain a reasonable number of examinees.) With items '*a", "b", and "c*' in 
Figure 2.5.1, there is substantial evidence of a misfit between the avail- 
able test data- and tljie estimated item characteristic curves (Hambleton,, 
19fiO) . It is surprising to note, given their apparent usefulness, that 
residuals have not received^ more attention from item response model re~ 

. searchers . 

Lord (1970, 197A) has advanced several app^roaches for addressing model- 
data fit. In 1970, Lord compared the shape of ICC curves estimated by 
different methods. In one method he specified the curves to be three- 

^ parameter logistic. In%the other . method, n6 ma'the^maticai form of the ICCs ' , 

• ' * ■■• , • . " ^ • • • ^' • 

was speci'f ied . Since^ the two" methods' gave very similar results, (see . > • • 

Fig~ure 2.5.2). he argued that it was reasonable to impose the mathematical 

form of three-parameter logistic curves on his data. Presumably Lord's 

study can be replicated on other data sets as well although his second 
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.Figure 2.5.1. Analysis of residuals. Possible Explanation: 

(a) failure to account for "guessing", (b) failure to account for 

''item discrimination'^ (c) biased item,* and (d) item fitted by the 
particular model. 
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Figure 2.5.2. Five item characteristic curves estimated by two 



different methods (reproduced from Lord, 1970) 
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method required very large examinee samples. .In a second study, Lord 
(1974) washable to assess, to some extent^ the suitability of ability 
-jestimates by comparing them to raw scores . The relationship should be • 
high but not perfect. / ^ . • . /. 

Simulation studies' have beep found to be of considerable value in 
learning more about item response models, /and hpw they compare in different 
applications (e.g., Hambleton, 1982a, 1982b; Hambleton & Cook, 198-2; Ree, 1979)/ 
It is possible to simulate data with known properties and see .how well 
the models recover the true parameters. Hambleton ancT Cook (1982) found, 
for example, when concerned with estimating ability scores for ranking, ^ 
description , or decisions , that the one-, two- , and three-parameter 
models provided highly comparable results except for low ability examinees^ 
Swaminathan (1981) conducted a study of Bayesian estimators 
and used a comparison of true and estimated difficulty values to evaluate 
these procedures (see Figure 2.5.3). 

Several researchers (for example, Hambleton & Traub, 1973; 'Ross, 
1966) have studied the appropriateness of different mathematical forms 
of item characteristic curves by using them, in a comparative way, to predict-- 
test score distributions (See Figures 2.5,.4 and 2,5.5). Hambleton and Traub • " 
(1973) obtained item, parameter estimaiies for the one- and two-parameter models from 
three aptitude tests. Assuming a. riormal ability distribution and ^u.sing 
test " characteristic curves obtained from both the one- and two-parameter ' ' 
logistic models ,' they obtained predicted score distributions |or each 
of the three aptitude tests. A goodness of fit index. was used to - 

o • 

compare actual test score distributions with predicted test score dis- ' 

tributions rom each test model. Judgment can then be used to determine 

the suitability of any given test model and the desirability of one model - 
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n=25 



N=^50 



BAYES 



r=.979 



I 3 
Iru* Otrflcult/ 



ML 



O -? -1 

O 



r=.927 



:ru# OlfHcuity. 



n=35' 



N=200 



4 



7 



7 ) 
Tfvt Olfflc«Ur 



r=.989 



\ ri=,;980 

Figure.^ 2.5.3. Bivariate plot of true and estimated values of 
difficulty (two-parameter model) r 



2 3 





Figure 2.5.5. Observed (•)' and ejipected 

(o) distributions for OSAT- 
Verbal using the one-parameter 
logistic model. 
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over another. Hambleton and Traub (1973) based their predictions upon a 
normal abildty distribution assumption however it is neither desirable ^ 
nor necessjary to/make such^an assumption- to 'ob/ain predicted ^core disi- 
tributions • 

, Finally, it is reasonable 'and desirable to generate testable, hypo these 
concerning model-data fit. Hypotheses- might be generated because they 
seem interesting (e.g.'% Are item calibrations . the same for examinees 
receiving substantially different types of instruction?) or because ques- 
tions may. .have arisen concerning the validity of the chosen item response 

^ ' * . . . • 0 , 

model and testing procedure(e . g . , T^afe- effect does the context in ^hich 
ah item is pilat-tested have on the associateii item parameter estimates?) 
On this latter/point, se^ for example, Yen (1980). Surprisingly, there 

; ■ • . ■ • \ . ^ 

is relatively little attention beyond the attention associated with 
dategory 1 and 2 for testing hypotheses. " ^ 

2.6 Summary , 

Our review: of relevant literature associated with conducting goodness 

'of fit studies revealed a substantial t^umber of appro^iches . From our \ 
perspective; however, there appeared to b'e too much emphasis on "statistical 
tests for determining goodness of fit* As an alternative, the* use of 
judgment in interpreting misfit statistics and other 'model-data comparisons 
for more than one model seems desirable. Perhaps the statistical approach 
can be replaced by the use of graphijcal method^, replications, cross 
validation techniques, study of residuals, baseline, results to aid '^in 

'interp? e.tations , study of practical consequences of misfit, and' so on. 



With respect to testing model assumptions, iinidimensionality is 
clearly the most impoxtant as sumption to s atisfy. Many test s of unidi- 

mensionality are available but those which are independent of correlations 

, ^ - - 

(Bejar) and/or incorporate the analysis of residuals (McDonald) . seem 

most useful. In category two, there is a definite shortage of ideas 

and techniques. Presently, plots of (say) item parameter estimates ob- 

tained in two groups are compared but- without the aid of, any "baseline 

plots." Or, statistical * tests are used to compare the two sets of item 

parameter estimates but such tests are less than ideal for reasons offered 

in section 2.2. Several new techniques seem possible and these will be 

introduced in the next chapter. In the third category, a number of very 

promising approaches have been described in the literature but they have 

received little or no, attention from researchers. Perhaps the problem 

is due .to a. shortage of computer programs to carry out necessary analyses 

or to an over reliance on statistical tests. In any case the problem is 

likely to be overcome in the near future and we will focus our attention 

in the next chapter on several of the more promising approaches in this 

category/ . ^ 
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3.0 Analysis of NAEP Mathematics Exercises 



3>1 Introduction 

In this section of the report (1) the NAEP mathematics exercises will 
be briefly described, (2) the particular mathematics exercises which were 
chosen for analysis will be described, and (3) the results from many item 
response model-NAEP math data fit investigations introduced in section 2" 
will be presented and discussed. 

3. 2 . Description of NAEP Mathematics Exercises 

In the 1977-78 NAEP assessment of mathematics skills of 9, 13, and 
17, yiear olds, approximately 650 test items (called "exercises" by NAEP) | 
at each age level were used. Available, test items at a given age level ^ 
were randomly assigned to one of ten forms. Each test form was adminis- 
tered to a carefully chosen sample of (approximately) 2500 examinees. ^ 
Elaborate sampling plans were designed and carried out to insure that' each 
form was administered to a nationally representative sample of ' examinees . 

Item statistics play ort'ly a minor part in NAEP mathematics test 
development. Test items are included in test forms if they measure what 
national panels of mattiematics specialists believe ^hould be included 
in the NAEP testing program. Content considerations are dominant in the 
item selection process. In this respect test development parallels the 
construction of criterion-ref erfenced tests (Pophafn, 1980; Hambleton, 1982c) 
Tables 3.2.1 and 3.2.-2 provide information on the distribution of item 
content across six content categories for four test booklets (two booklets 



at the^ 9 and 13 year old levels). Math calculations, story problems, ! 
and geometry appear to be the -most frequently occurring types of test 
items'. ' ■ ■' 
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Table 3.2.1 



Content Classificat ion . Summary of NAEP 
Math Booklet No. 1 and 2 Test Items 



Booklet 1 



Booklet 2 



Story Problems 

Money * 
General 

Logic , Probability, 
Permutation c\nd 
Combination 

Total ' 



1 
5 

4 

10 



Story Problems 

Money 
General 

Logic, Probability, 
Permutation and 
Combination 

Total 



3 

2 

7 
12 



Geometry 
Story 

Definition/ Operations 
Figure Interpretations , 
Manipulation i^. 

Total 



0 
9 

5 
14 



Geometry 
Story 

Definition/ Ope rat ions 
Figure.- Interpreta^tions , 
Manipulation 

Total 



0 

9^ 

1 
lO" 



Definition 
Total . 



Definition 
Total 



16 



Calculation 

General 
Algebra . 

Total 



15 
23 



Calculation 

General 
Algebra 

Total 



\ 



25 
26 



Measurement 

English 
Metric 

Total 



3 
3 



Measuremeiit 

English 
Metric 

Total 



1 

4- 



Graphs and Figures 
Total 



Graphs and Figures 
Total 



ERLC 



43 



-AO- 



Table 3.2.2 

Content Classification. Summary of NAEP 
Math Booklet No. 1 and 2 Test Items 
(13 Year Olds, 1977-78) 



Booklet 1 



Bpoklet 2 



St )ry Problems 

Money » 3 

Cirneral .6 
Logic, Probability , 

^Permutation and '5 

Dombination 

Total 14 



Story Problems 

Money 
General 

Logic, Probability , 
Permutation and 
Combination 

Total 



2 
9 

4 
1.5 



Ge ome t ry 
S tory 

r ef inition/Operations 
l igiire Interpretation, 
Manipulation 

'Jotal 



1 
9 

\ 3 



13 



Geometry 
Story 

Def inition/Operations 
Figure.^ Interpretation, 
Manipulation 

Total 



1 
7 

2f 
lOi 



^ )ef inltion 
Total 



Def^inition 

\ " 

Total 



C alculation 

General 
Algebra 

Total 



14 
1 

■15 



Calcul at/ion 

General 
Algebra 

Total 



17 
5 

"22 



Measurement 

English 
Metric 

Total . 



Measurement 

English 
Metric 

Total 



1^ 
0 



Cr ap. I IS and I'lgurcs 
Total 



Graphs and Figures 
Total 
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\ About 50% of the tes't: items in the ISll-^lQ assessment were included 

^ . — on— the— p^evA-cms— NA&P-TOatrhema-t- ics a3s e&sme^H^-^^n--d-97^^^-7^2^ — In— ati dit io n » o 



the 1977-78 assessment some test Items were included In the mathematics 
test booklets at all three age levels. While in our research investigation 
"linking'* test items across age levels to a common scale was of no inter- 
est, such a task could have been accomplished with the aid of these 
common test items (Lord, 1980;' Wright & Stone, 1979) . However, at a 
given age. level, there were no common test items. Had we been interested 
in "linking'* test items at a given age level to a common scale, the task 
could have been achieved easily because of the plausible assumption that 
test forms were administered to equivalent? ability groups (Lord, 1980; 
Hambleton & Swaminathan^. 1982) . 

Test items in th^ NAEP mathematics assessment were of two types: 
multiple-choice, and open-ended. Tables 3.2.5, 3.2.4, 3.2.5, and 3.2.6 
provide information on the item formats and content categories of test 
items in NAEP math booklets 1 and 2 for 9 and 13 year olds. Among the 

^ ■ \ ^ 

multiple-choice test items it was also interesting to note that the numbeiT 
of answer choices varied. Information reported in tne four tables pro- 
vided the basis for several important analyses described in section 3.6. 

3.3 Description of Data -t*... 

Six NAEP mathematics test booklets from the 1977-78 assessment were 
selected for analysis: ' « 

9 Year Olds 

Booklet No. 1, 65 test items «^ 

Booklet. No. 2, 7S test items ^ . 

Booklet No. 3, 68 test items 



'ERIC 



4 



. -A2- 
Table.-3.2.3 



/ 

/ 

Format and Content Classification of NAEP 
Math Booklet No. 1 Test Items 
(9 Year Olds, 1977-78) 





Item No,. 


Answer Format ■'- 


, — — ——— & 

Category 




■ . 1/102A 


-MC 


Definition 




• 


2/i02B""»~ 


, '" MC (6 options) 


Definition 






3/103A 


MC 


■ Story problem 


- money 




A/1'04A 


MC (6 opfiions) 't,- 


Geometry - definition 




5/lOAB . * • ' ■ 


MG (6 options) 


Geometry - definition 




6/105A 


MC 


Geometry - figure manipulation,- 
interpretation 




7/106A 


OE 


Geometry - operations 




8/106B 


OE 


Geometry - operations 




9/106C 


MC 


Geometry - operations 




10/107A 


MC (6 options) 


Measurement - 


English 




• 

11/108A 


OE 


Calculation 


t 




12/108B 


OE 


Calculation 




• 


137108C . 


OE 


Calculation. 






1A/108D 


OE 


Calculation 






r5/io«E 




Calculation 






\ ■ 16/108F 


OE 


Calculation 






17/109A 


MC 


Story problem 


- logic 




18/llOA 


OE 


Story problenj 


" general 




19/lllA 


MC 


Geometry definition 




20/112A 


OE 


Calculation 




• 


2l/ri2B 


OE 


Calculation 


■ ' if 




22/113a" 


MC* 


Measurement - 


English 




23/114A 


MC (6 options) 


Story problem 


- general 






OF 


Calculation - 


algebra 


• 


25/115B 


■ OE 


Calculation - 


algebra 




26/115C 


OE 


- Calculation - 


algebra 




27/115D - 


. OE . 


Calculation - 


algebra 


• 


28/115E 


V OE 


Calculation - 


algebra \ 




29/ll5^F 

3o7ii5n 


OE 


Calculation - 


algebra 




OE 


Calculation - 


algebra ' . 



MC Items have 5 answer choices 



(inb^luding. *M don't know") unless otherwise noted. 
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Table -3.2.3 (continued) 



Item No, 



Answer Format 



Category 



31/116-A,, 
' 32/117A 
33/117B 
3A/118A 
35/119A 
36/120A 
37/120B 
38/121A 
39/122A 
40/123A 
41/124A 
42/125A 
A3/125B 
44/125C 
45/126A 
46/127A 
47/128A 
48/129A 
'49/129B 
50/130A 
5l/r30B 
52/131A 

53/131B 

54/131C 

55/132A 
56/133A 
.57/134A 
58/134B 
59/134C 
60/135A 



MC (6 options) 

MC 

MC 

MC (4 options) 
MC (6 options) 

OE \ 

OE 

MC (10 options) 
MC (6 options) , 
MC 
OE 

OE ' 
OE 
OE 
OE 

MC (4 options) 
MC (4 options) 
MC 
MC 

MC (4 . options) 
MC (4 options) 
MC (7 options) 

MC (7 options) 

MC (7 options) 

OE 
OE 

MC (6 options) 
MC (6 options) 
MC (6 options) 
OE 



Graphs and Figures . 

Definition 

Definition 

Measurement - metric * 
Graphs and Figures 
Calculation ; 
Calculation 
Definition 

Story problem -general 
Calculation 

Story problem - general 
o Calculation ? 
Calculation 
Calculation 

Measurement - metric . 

Calculation - algebra 

Measurement - metric ■. ^ 

Graphs and Figures. 

Graphs and Figures 

Story problem'- logic 

Story problem - logic 

Geometry - figure manipulation, 
interpretation ^' 

Geometry - figure manipulation, 
^ interpretation 

Geometry - figure manipulation, 
interpretation 

Graphs and Figures ^ 

Story problem - general 

Geometry - definition 

Geometry - definition 

Geometry - definition . « 

Story problem - probability • 



ERLC 
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Item No. 



Table 3.2.3 (continued) 



Answer Format 



Category 





61/1 36A 


OE 


Measurement -English 




62/137A 


OE 


Definition 




.63/138A 


OE 


Calculation 




6A/139A 


MC 


Ge^ometry - figure manipulation, 
interpretation 




* 65/lAOA 


MC 


Definition 



' 4 



TRIG 
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Table 3.2.4 



Format and Content Classification of NAEP 
Math Booklet No. 2 Test Items 
(9 Year Olds, 1977-78) 





Item No, 


Answer Format 1 

.... 


Category 


1/202A . 


MC 


Definition 


. 2/2Q2B 


MC 


Definition 


3/203A 


OE ^ 


Calculation 


4/203B 


OE 


Calculation " 


5/203C 


OE 


Calculation 


. 6/203D 


• OE 


Calculation 


7/203E^ 


OE 


Calculation 


8/203F 


OE 


Calculation 


9/204A 


OE 


Calculation ^ 


10/204B 


OE 


Calculation 


11/204C 


OE 


Calculation 


12/204D 


OE 


Calculation 


13/205A 


MC (6 options) 


Geometry - operations 


14/206A 


MC (6 options) 


Story problem - moitey 


15/207A 


MC 


Graphs and Figures 


16/207B 


MC 


Graphs and Figures 


17/208A 


OE 


Calculation 


18/208B ' 


OE 


{Calculation 


1Q/208C 


OE / 


Calculation 


^ 20/209A 


MC 


Story problem - combinations 


21/210A 


MC (8 .options) 


Graphs and Figures 


22/210B 


MC (6 options) 


Graphs and Figures 


23/210C 


MC (9 options) 


Graphs and Figures 


24/211A 


MC (4 options) 


Definition 


25/211B 


MC (4 ojptions) 


Definition 


26/211C 


MC (4 options) 


. Definition 


27/211D 


MC (4 options) 


* Definition 


28/211E 


MC (4 options) 


Definition ' 


29/212A 


MC (4 options) 


Measurement - metric 


30/212B 


MC 


' Measurement - metric 


"MC Ttems have 


5 answer choices (including "I don' 


t know'O unless otherwise noted. 



Table 3.2.4 (continued) 



Item' No. — ^ Afl s wer For mat — C a teg o4:-y- 



Q 1 / O 1 OA 


. . . ■ 


CSV 






PoT r»iiT ot"*! f\Y\ — lal cy^V^T*i3 




5/,/ 
















M 


nv 




















. jjei. -Ln XL xon 




3-)/ ZiDC 




Ub 






ueiini L ion 




30/ Z±OA 




MP 


KO 


op Lion s ^ 


VJcU IIlc L I. y tl c X Xll X L-LU 11 




3 / / ZlOD 




Mr* 




opt ions ) 


oeoTueLLy ueix^ixLxvjn 




o o / o 1 

3o/ zloO 




"MP 




opt ions ) 


oeoTTie Liry atixxiixLxuii 




39/ 21 /A 




ML 






Story problem " money 




/ / O 1 OA 

40/ 21oA 




Ub 






Calculation 




41/ 218B 




At? 






Laicuiation a 




42/218C * 




OE 






La.icuia Lion •< 


a 


43/2ioD 










Lia,XCUXclLXUii 




44/ 21oE 


t 


ub 






L a 1 c u 1 a L 1 o n. 








ub 






LiciX CUXd L XUll ^ 




A ^ / O 1 Q A 

^0/ z±yA 


\ 


MP 






C U iliC L 1. y U r ' ^ J-VJliO 




4 / / ZZUA 




ub 






LiciXCUlcl L XUIl 




An/ ZzUd 




Ub 






LicllCUlcl L XOIl , ^ 








ur* 






• 

PoT o+"*i f^Y\ 

L<axcuiciL xun 




50/ /ziA . 




MP 

IrlL 






ueomeij^y ucixiixlxuh 




C 1 / o o o A 

5 1 / 222A 




"MP 

ML 






Measurement *■ metric 

/ 


/. 






ML 






. ije L ini c ion 




53/224A 




MC 






Definition 




54/224B 










Definition 




55/225A 




MC 






Story problem - logic 




56/225B 




MC 






Story problem - logic 




bllllbZ 




MC 






Story problem - logic 




58/226A „ 




MC 






Story problem - general 




59/226B 




MC 






Story problem - general : 




60/227A 




MC 






Calculation " 





0 



50 





.■■■■■) 


J . ■. • ■ ■ 

. • * • I •'■ 't ,'■«.;■'..' 1 








a 


. • ■ . .* ' '. 
• ^ '• ■ ' 

. 1-? 
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Table 3.2 


A (continued) 


• 




Item No . 


■ Answer Format 


Category 




* 












61/228A 


MC 


Geometry - definition- . . 






62/228B 


MC 


Geometry - definition 


• 




63/229A ^ 


• MC ■ 


Definition 






•6A/229B 


MC 


Definition 






65/229C' . 


MC 


Definition - . . 






66/2 30A 


OE ■ 


Caik^ulation \. ^ 






67/231A 


, OE 


Story problem - money . 






68/232A 


OE 


• . , Geometry operations 


• 




69/233A. 


MC i! 


Stor^'^problem logic ' 






70/234A 


OE 


Story problem - probability 


• 


) \ 


71/235A 


OE '. 


geometry - figure manipulation, 
interpretati'on 






\ 72/236A 


OE . 


Calculation 






\ 73/237A. 


OE 


Measurement - English 






\4/238A 
- 7^(239A 


OE 


Graphs and Figures . 




MC 


Measurement - metric ^ 


% 




\ 

■ ■ . ^ \- 

' \ 

\ 

\ . 


p 












■ ^ 






- 


\ 

\ . 








. / 


\ 

\ 










\ 

\ 

\ 






'eric 









^ Table 3.2.5 

^ i ' ' ' ■ ' 

Format and Content Classification of NAEP Math Booklet^ ^ - " ; \ ; 

:No. Idlest Items ' ^ ^ «r 

* (13 Year Olds, 1977-78) . . ' ' ' / . . ; 



Item No., Answef Format 1 Category 







OE 




Story problem - money 


2/103A 




MC 




Definitions 


3/103B 




MC 




Definitions 






OE 




Measurement - English 


■5/105A 




•MC 




Calculation , 


6/106A . 

A- 

7/106B 




MC 




Geometry - definition, operations 




MC 




Geometry definition, operations' 


.8/106C ' 




MC 


'■■ * 


Geometry .definition, operations 


.9/107A 




MC 


• 


Story probleint - logic 


10/108A ' 




MC 




.Measurement - metric i 


11/X09A 




-.OE 




Calculation - subtraction^ ^ 


12/109B 




OE 




Calculatio'n - subtraction 


1^/109C 




OE 




Caluclation -. subtraction** 


1471090 ■ . ■ 




OE 




Calculation - subtraction 


15/109E 




OE 




Calculation - subtraction * 


16/lb9F 




OE 




Calculation - subt^actioii 


17/llOA 




MC 


(4 options) 


Measurement - metric 


18/lllA 




OE 




Story problem - general 


19/lllB 




OE 




Calculation 


20/112A 




■ OE 


- 


. ^Calculation 


21/112B 




,0E 




Calculation 


.22/113A 




MC 


(10 options) 


Definition 


23yil4A 




MC 




Definition 


24/114B 




MC 




Definition 


25/115A 




OE 




Story problem - money 


26/116A 




MC^ 




Geometr>f - definitions, operations 
Geometry definitions, operations 


27/1-16B " 




MC 




28/117A 


K ■ • ■ 


OE 




Geometry - definitions 


29/118A 




OE 




Measurement - English 


30/119A 




• MC 


(7 options) 


Story problems^ - general 



MC items have 5 answer choices (inclutiing "I don't know'^) unless otherwise noted, . 



■ l' ■ 


.A. 0 .9 . 
.-^ " ' ■ ' / 




1 , , ... - , ^ . , ., 

.., •■.■ "»'■ 


- ^- 
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Table 3.2.5 (continued) 




Item No. 


Answer Format 






31/120A 


, " MC 


Geometry ~ figure manipulation. 


• 






interpretation 




32/120B 


MC 


Story problem - general - " 




.33/121A 


MC (6 options) 


Story problem - general 




34/122A 


MC • 


Geometry - definitions 


• 


357122b" •; 


MC 


Geometry - definitions 




36/123A 


MC 


Story problem - money 




37/124A 


MC (6 options) 


Geometry - s tory^ problem * • . 


• 


; 38/125A . . 


MC 


Definitions * ^ • 




■ 39/126A 


MC 


Definitions ' \ , 




40/127A 


MC ' 


Story problem - combinations 




'41/128A 


MC ■ ^ ' 


Definitions . ' 




42/.129A 


MC X6 options) 


Geometry - definitions, operations 


• 


43/l'30A 


MC 


Geometry - figure manipulatid'n 




44/131A 


OE ' . 


Calculation » ' '\ 




45/131B 


' OE , 


Calculation 




46/132A 


MC 


Story problem - general 




47/133A 


MC . 


G#omet*ry - story problem . ^ ^ - ; 




48/134A . ^ . 


MC (6 options) 


Definitions 




49/135A 


OE 


Calculations - algebra ' 




50/ 13^ 


MC 


Story problem - general 


• 


51/137A ^ 


MC ( 6 options) 


Story^problem - ""probability . 




52/137B . ' ^ 


MC ( 6 options) . 


Story problem ^*^robability 




53/a38A 


MC ( 6 options) 


Gebmejtry - figure manipulation 




54/139A • 


. . OE . 


Calculation. 




55/140A 


OE 


Graphs and figures • 




56/141A 


^ MC . • 


Story problem - logic 




57/142A . . 




Measurement — ^ English 




58/143A^ 




Calculation 


# . 


















* 








r 


« 




1 




■ . ... ... V . .. 1 . . , 


• ■ * * 



Table 3.. 2. 6 



Format and Content Classif ieation of NAEP 
Math Bodklet No. 2 Test '^ems 
Via" Y6ar Olds, 1977-78) 



Item No. 



Aniswer Forma t-'- 

\ 



Category 



17202A,. 
2/203A ■ 
• 3/204A 
4/.205A 
5/206A 
6/207A 
7/208A 
8/209A 
9/210A 
10/210B 
11/210C " 
12/210D 
I 13/211A 
/ 14/212A 
15/213A • 
16/21^^ 
17/214B~ 
•18/214C 
19/214D 
20/214E 
21/214F 
Z2/215A 
23/216A 
24/2i6B 
25/216C 
26/217A 
27/217B 
28/2J.8A 
29/219A 
•30/22PA 







6 








4 






McV 






MC 


\ 




• OE 






OE 






OE 






OE 






OE 






OE 




; 


OE 






MC 


(6 


options) 


MC 






MC 


(6 


options) 


OE 






OE 






•OE 






OE 






OE 






OE 







MC. (6 options) 
OE 

OE . • 

OE 

MC ' 

MC 

OE 

MC 

OE 



Calculation - algebra 

Calculation 

Calculation 

Story problem - logic 

De;:initionsS 

Graphs and Figures 

Measurement - English 

Stiory problem - general 

Calculation 

Calculation 

Calculation • 
Calculation 

Geometry definitions 

Calculation - algebra 

Geometry - story ptoblem 

Calculation 

Calculation 

Calculation 

Calculation 

Calculation 

Calculation 

Geometry - definitions 

Calcula'tion 

Calculation 

Calculation 

Geometry - definition 

Geometry - definition 

Story problem general 

Story TD^oblem - money 

Story problem - probability 



"^MC It^ems have 5 answer choices (including **I don't ktiov'*) unless otherwise noted 
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Table 3.2.6 (conrinued) 



Item No. 



Answer Format 



Category 



■eric 



31/221A 
32/222A 
33/222B 
347223A 
35/224A 
36/225A 
37/225B 
38/225C 
39/226A 
A0/227A 
41/228A 
42/228B 
. A3/229A 
44/230A 

,45/231A. 

46/232A 
.4I/232B 
48^233A 
49A233B 
50/233C 
51/234A 
52/234B 
53/235A 
54/236A 

_557237A- 
56/238A 

.57/239A 
58/240A 

' 59/240B 
60/240C 
61/241A 
62/241B 



■ MC ' 
MC (4 options) 
MC (4 options) 
MC (6 options) 
OE 

MC (6 options) 

MC (7 options) 

MC (6 options) 

OE 

MC 

OE 

OE 

MC (4 options) 
MC 

MC ( 6 options) 

MC 
MC 
OE 
OE 

OE 

MC ( 6 options) 
MC ( 6 options) 
OE 
MC 

MC (6 optioln 
OE 

MC (6 options) 
MC (6 options) 
MC (6 options) 
MC (6/ options) 
OE 



\ 



7 



Definition 
\Definition 
Definition 

Story problem - general 

\ 

'Story problem - money \ 

Graphs and figures 

Graphs and figures 

Graphs and figures , 

Calculation - algebra 

Story problem - general 

Calculation - algebra 

Calculation - algebra 

Story problem - general 

Geometry - figure manipulation, 
interpretation 

Story problem -^perml^f>^^ionl 
and coiiibdrnation 

Story problem - general 

Story problem - general 

Definition 

Definition 

Definition 

Geometry - definitionrS 

Geometry - definitions 

Story problem - general 

JJeometry - figure manipulation, 
interpretation 

feometry / definitions, operations 

Story problem - general 

Story problem - probability 

Graphs and figures 

Graphs and figures 

Graphs and figures 

Calculation - algebra 

Calculation - algebra 
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, 13 Year Olds 

Booklet No. 1, 58 test items 
Booklet No, 2, 62 test items 
Booklet No. 3, 73 test items 



In some of the computer printouts which follow the six brooklets above are 
desiftnated 109, 209,^309, 113, 213, 313, respectively. There was no 
particular pattern t^o our choice of data sets for the various analyses. 
For some analyses all six data sets were used, for others, only one or 
two were used. 

Tables. 3,3,1 ^^nd 3,3,2 contain the one- and three-par^ameter logistic 
-model ^parameter estimates for items in the six NAEP math booklets men^ 
tioned- above, B^t^een 2400 and 2500 examinees were used in item parameter 
estimation which was carried out with the aid of LOGIST (Wingersky, 1982; 
Wingersky, Barton, & Lord, 1982^),' 



\ 



TRIG 



^The n/( 
of the comfiu 




t recent references to LOGIST are given but the 1976 version 
ter program was used in our analyses. 
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- , Table 3.3.1 

NAEP Math Item Response Model Parameter Estimates 
(9 Year Olds, 1^7 7-78) 








Booklet 


No. 1' , 




♦ 


Booklet 


No. 2 






Booklet No. 3 




Test 




























Item 




1-p 




3-p 










3-p 








3-p 






j; 


b 


b ' 


a 


c 


b 




b 


a 


c 


b 


'b' 


A 

a 


c 


.1 




-.22 


. 20 


1.15 


.19 


-1. 39 


-2.86 


.24 


. 09 


.44 


2.67 


.08 


•««^- 
. 01 


2 


.17 


. 30 


1 . 20 


. 09 


-1.40 


-2 


.82 


. 25 


. 09 


- . 31 


-1. 76 


.09 


. 01 


3 


-. 22 


.15 


1 . 20 


.17 


-2.63 


-2 


.36 


,11 


.09 


.15 


.24 


.27 


. 01 


4. 


-2.55 


-4. 01 


. 37 


..06 


-z . 13 


-1 


.64 


. 99 


. 09 


-.14 


. -.54 


. 15 


. 01 


5 


-2 . 33 . 


-3.39 


.40 


• 06 


-2.21 


— 1 


• OD 


.85 


.09 


-1.58 


-4.96 


.17 . 


.01 


6 


-.93 


-1. 77 


. 27 


. 06 


/ —1 . ^ U 


-1 


.07 


. "4 


. U9 


-1 . z 3 


-1.24 


. .64 


. 01 




2 . 18 


1.-91 


1. 56 


' *07 


1. 99 


-1 


.49 


1 . 05 


. 09 


J . 62 


3.22 


.76 , 


. 01 




.82 


.86 


.70 


. 03 


-1 . DO 


-1 


26 


1 . 00 


.09 


. -1.49 


-2.31 


.37 


. 01 


9 


.21 


. 38 


. 51 


. 05 




- 


10 


T CO 

1 . 5o 


. 12 


. 93, 


48 .36 


.01 . 


. 01 


10 


. 53 


. 58 


1.13 


.08 


-.48 




20 


1.42 


.09 


-1.63 


-1.97 


.50 


.01 


11 


-2.32 


-^1. 62 


1. 42 


. 06 


. 01 




.09 


1.48 


.05 


.04 


. 69 


1.22 


.27 


12 


-1.81 


-1. 30 


1. 26 


.06 


-. 01 




.08 


1.26 


. 05 


-1.94 


-5.11 


• 21 


. 01 


13 


-2.17 


-1.48 


1.55 


. 06 


2.63 


2 


.18 


1.07 


.02 


.83 


.70 


.82 


.01 


14 


-1.13 


-. 79 


1. 24 


. 06 


* 60 




.71 


.95 


>09 


1.65 


1.52 


•71 


. 01 


15 




-1.09 


1.49 


.06 


-1.13 


-1 


. uu 


.66 


.09 


1.15 


.81 


1. 14 


.00 


16 


-1.20 


81 


1. 39 


. 06 


32 




.06 


1.03 


.17 


, 6 . 31 


4.83 


.88 


.00 


\ • 


.19 


. 51 


. 32 


. 06 


-1 .lo 




93. 


.82 


. 09 


. 31 


.29 


.89 


.02 


i) 18 


-i:64 


-1.63 


.65 


. 06 


-1.14 




.88 


.83 


. 09 


-.02 


- -.05 


.53 


.01 


19 


-1.90 


-2.23 


.52 


. 06 


-. 26 




07 


. 99 


.09 


-1.50 


-1. 28 


.88 


.01 


20 


-.60 


-.47 


.79 


.06 


1.99 


1 


.64 


1.07 


.03 


-2.00. 


-1.69 


.89 


.01 


21 


.48 


. 37 


1.19 


.01 


- .60 




33 


1 . 06 


.09 


-2 . 73 


-4.04 


.'40 


. 0^1 


22 


1. 55 


1. 25 


1.46 


.06 


-. 32 




10 


1.19 


. 09 


-1.58 


-l.S^ 


^-.66 — 


. oX^ 


23 


-.14 


. 32 


1. 39 


. 21 ' 


.52 




.49 


1.25 


..05^ 


-'-lT'6'4'" 


-i.64 


.67 


. 01 


24 


-1.69 


-1. 22 


1. 20 


.06 


\ -3.5P_ 


, 2' 


r87- 


. 96 


. 09 


-1.27 


-1.6Z 


.47 


. 01 


25 


.49 


.4 3 






's^3 . 4 8 


-3 


.12 


O 1 

.81 


. 09 


o o 

- . 28 


,-.21 


1.02 , 


m 
. Ul 


-'26 




. 06 


.10 


1.35 


.03 


-3\62 


-2 


.9,3 


1.00 


. 09 


-1.91 


-1.51 


1.05 . 


.01 


27 


-.82 


-. 56 


1.06 


.06 


-3.03 


-3 


07 


^•.65 * 


. 09 


-1.75 


-1.34 


1.11 


.01 


28 




.68 


.60 


, .86 


.02 


-2.6K 


-2 


.71 


.64 


.09 


-1.17 


-.85 


1.18 


.01 


.29 


-1.35 


-1.07 


.93 


' .06 


1.58 


\ « 2 


.45 


1.10 


.16 


-.81 


-.62 


1.05 


...01 


30 


-.83 


-.53 


1.33 


.06 


1.55 


\ 2 


38 


.56 


.09 


-1.41 


-1.06 


1.14 


.01 






















-■3 









\ 

u 
o 
I 
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. ^ Table 3.3vl (continued) 
NAEP Math Item Response. Model Parameter Estimates 
"* . C9 Year Olds, 1977-78) 



Test 
Item 



Booklet No» 1 







3-p 




b 


b 


a 


c 


-.89 


-.66 


.98 v' 


.06 


3.59 


2.51 


2.00 


.02 


1.61 


8. 93 


.12 


.06 


-.64 


-.63 


.56 


.06 


-1.51 


-1.41 


.71 


.06 



Booklet No. 2 



1 -n 
J. p • 




J P 


b 


b 


a 




1 06 

A. u \J\J 


. 80 


1.10 


.88 


.94 


.03 


.18 


1.15 


- . 10 


.08 


1 .13 


1 3 


.27 


1. 10 




1 . 38 


.87 


•1.87 


-1.74 


.69 


1. 60 


1.68 


.91 






1 7Q 


m \J A. 




1 09 


• *4D 






X . U / 


• D J 


1 'X^ 


-.25 


-.15 


1.40 


•1.20 


-.74 


1.42 


.18 




_ 1 . 21^ 


1.58 


1. 77 


1.15 


■2.37 


-3.15 


.44 


•1.23 


-.93 


.88 


.00 


.16 


.85 


.39 


.58 


.75 



Booklet No. 3 



1-p 
b 



3-p 



31 
32 
33 
34 
35 

36 
37 
38 
39 
40 

41 
42 
43 
44 
45 





.68i| 


-.51 


.89 


.06 


2 


.47 ' 


1.88 


1.00 


.00 


-2 


.99 


-3.93 


.47 


.06 


1 


53 


2.52 


.67. 


.13 


1 


77 


2.40 


1.00 


.13 



1.55 


1.07 


1.33 


.00 


1.77 


1.58 


.77 


.00 


"4.02 


2.53 


1.40 


.00 


4.90 


2.80 


1.62 


.00 


1.03 


2.56 


.27 


.06 



46 


.69 


.96 


1.18 


.16 


47 


-.18 


-.15 


.37 


.06 


48 


-1.74 


-1.51 


.8*2 


.06 


49 


1.02 


1.14 


1. 38 


.13 


50 


-.72 


-1.48 


.24 


..06 



51 


1 . 23 


3.82 


.21 


.06 


52 


.71 


' 3.32 


.14 


.06 


53 


2.49 


2.49 


1. 51 


.07 


54 


2.55 


2.39 


1.17 ' 


.06 


55 


5.06 


3.11 


1.44 


.00 


56 


2.14 


1.60 


1.05 


.00 


57 


•12 


.48 


.82 


. 15 


58 


I. 32 


1.33 


1.08 


.09 


59 


1.06 


1.20 


1.62 


. 15 


60 


1.51 


. 1 . 55 


.63 


.01 



-1.49 
-1.04 

-.01- 
.28 

1.20 

-.17 
.60 

-. 74 
.01 

1.33 



-3.33 
-.63 
.39 
.73 
2.30 

.02 
1.21 
-.56 

.29 
1.34 



.23 
1.27 
1. 44 
.76 
.60 

.45 
.65 
.73 
.94 
.92 



-l.OQ 
4.23 
.91 

-2.06 
-.26 

-1.11 
1.68 
-1.47 
-3.45 
-4.21 



-.75 
3.57 
1.41 
-1.97 
-. 97 

-.94 
■1.45 
-1.71 
-4.02 
-5.24 



1.10 
.85 
.77 
.73 
.15 

.87 
*.79 
.54 
.58 
.60 



.01 
.01 
.15' 
.01 
.01 

.01 
.01 
.01 
.01 
.01 




2.08 


1.88 


.73 


JOl 


1.51 


1.22 


1.09 


•.04 


-2.62 


-2.48 


^^76" 


.01 


-2.58 


-2.15 


.98 


.01 


-2.41 - 


-1.95 ■ 


1.05 


.01 


-2.05 


-1.75 


.90 


.01 


-2.21 


-1.96 


.85 


.01 


-2.22 


-1.91 


.90 


.01- 


-.16 


-.27 


.39 , 


01 


1.98 


3.37 


.69 


.12 


-1.43 


-1.27 


.82 


.01 


-.86 


-.84 


.69 


.01 


-.10 


-.25 


.28 


.01 


1.36 


2.02 


1.01 


.17 


-.96 


-1.05 


.59 


.01 



- ■ . Table 3-3,1 (continued) 
NAEP Math Item Response Model Parameter Estimates 
(9 Year Olds, 1977-78) 







1500 rClc^t 


No . -L 






Booklet 


No . 2 






■ - 

Booklet No 


. 3 




Toe 








* ■ 


















X te m 










1-p 








1-P 




3~p 






b 


b 


a 




b 


b 






K 




d 


c 


\J J. 




1 88 






— 9 1 ft 


. 00 


.09 


.09 


. 73 


.78 


1.08 


.09 


62 


-1.95 




. u u 


OA 


'^O • 


Q Q 
. O J 


.35 


.09 


-.07 


.18 


1.42 




.09 


1 7 


. ^ '4 


OA 


1 A Q 


Z .q J 


1 .ZD 


.18 


.43 


-68- — 


-r3"3 


■ .01 




70 


Q Q 


71 






2.49 


1.79 


.12, 


--^"I'.^l 


1.27 


.94 


.06 


65 


1.12 


3.31 


.22 


.06 


, 1.21 


. 4,72 


.19 


' .09 


-1.41 


-1.39 


. 68 


.01 


U D 










z . Iz 


2.14 


.75 


.03 


-.51 


-1.03 


.28 


.01 












1 . 1^ 


1.45 


.68 


.09 


.70 


^ .56 


.91 


.01 


68 












. 3.15 


1.23 


.00 


-.66 


-.84 


.49 


.01 


69 










OA 


. 35 


.46 


.09 










70 










2.61 


-4.84 


.38 


.03 








• 


71 ' 










2.88 


3.16 


.65 


.02 










72 










3.28 


2.38 


1.14 


.00 










73 










.74 


1.15 


.57 


.09 










74 










.49 


.64 


.80 


.09 










75 










.73 


1.51 


.39 


.09 











ERIC 



6i 



2 



; . Table 3.3.2 

NAEP Math Item Response Model Parameter Estimates 
(13 Year Olds, 1977-78: 



Test 
Item 



Booklet No. 1 

l^p ' 3-p 

b b a 









Book! e.t 


No . 2 






Eonklph No 


o 






1 


— n 






3-D 


• 


1-D 




3-D 

if 




c 




b 




b 


a 


c 


b 


b 


a 


c 


.11 




. 32 * 




.lb 


71 




-1 . 31 


- . 86 


1 .52 


.11 


.11 




. 10 




O Q 

. Zo 


T in 


. 10 


-^L . DO 


-1 .00 


1.48 


.11 


.11 






_ 


.77 




. 04 


-1 . 1 J 


-1.27 


.4 3 


.11 


;04 






-1 


.07 


A Q 


. 04 


39 


-.14 . 


.8.7. 


. 11 


.11 


- 


.30 




. 02 


1.13 


.13 


.39 


-.39 


1.21 


.06 


. 17 


1 


56 


1 


. 58 


7 9 


. 00 


-2.63 


-3.12 


/ o 

. 48 


. 11 


. 21 




. U J 




. 05 


7Q 


. 04 


79 


-.47 


• 90 


.11 


. 11 


-3 


..13 


-3 


.51 ■ 


. D J 


. 04 


-2 . 06 


-1.58 


o c\ 

.89 


. 11 


.11 


-1 


.78 


-1 


.25 


l.DD 


. 04 


1 . 51 


2. 21 


1.00 


. 16 


.11 


-1 


. 72. 


-1 


. 21 


1.65 


.04'- 


1.29 


2.59 . 


1.21 


.22 


.11 


-1 


.61 


-1 


.15 


I.JO 


. 04 


-2 . 07 


-1.73 


. 77 


.11 


.11 


-1 


.40 


-1 


. 01 


1 A Q 


. 04 


. 84 


.60 


1. 10 


. 00 




-2.51 


-2 


.35 


Q 1 
. O J 


.04 


. 15 


.16 


1,15 


. 03 


. 11 




.33 




.52 


^ A 


. 04 


-2 .29 


-4:-06 


. 29 


.11 


.11 




.91 




.90 


.99 


.04 


-.04 


.45 


1.16 


.21 


1 1 


-2 


16 


-2 


. 06 


"7 Q 


. 04 


. 19 


.61 


1.15 


.20 


1 1 


-1 


78 


-1 


. 61 


. ob 


.04 


. 31 


. 70 


1. 32 


. 20 


m 

• W X 


-1 


89 


-1 


71 


. O / 


. 04 


1. 06 


2.19 


2.00 


.26 


. 11 


-3 


03 


-3 


75 


c: c 
. 33 


. 04 


-1 . 08 


^ -.66 


1. 12 


.11 


.04 


-3 


03 


-2 


o / 

.84 


.84 


.04 


.69 


.86 


1.24 


.14 


. 02 


-2. 


89 


-3 


/5 


. 0 L 


. 04 


— . JO 


- . 10 


. 98 


. 11 


.11 


2 


70 


J 


5 3 


z . uu 


. 06 


O Q A 
— Z . O^ 


-2 .61 


. 71 


. 11 


. 25 




53 




44 


1 1 Q 

1 . lo 


. 01 


o r\r\ 

- J . 00 


-2.48 


.86 


. 11 


.16 




22 




23 


1 1ft 
1 . Lo 


. 03 


O OA 


-2.40 


o o 
. oo 


. ii 


.11 




48 




40 


.78 


.04 


" -3.30 


-2.68 


.92 


.11 


.11 


-2 . 


02 


-6 


17 


.19 


.04 


-3, 37 


-2.67 


.97 


.11 


.02 




19 




23 


. 32 


.04 


-3.12 


-2.43 


.99 


.11 


.15 


1. 


19 


1. 


24 


1.39 


.10 


.14 


.47 


.84 


.14 


.02 


• -2. 


39 


-2 


57 


.66 


.04 


1.62 


2.02 


.82 


.11 


.15 


1. 


67 


, 1. 


.71 


.71 


.00 


.32 


1.16 


.30 


.11 



1 

2 
3 
4 
5 

6 
7 

-8 
9 

10 

11 
12 
13 
14 
15 

16 
17 
18 
19 
20 

21 
22 
23 
24 
25 

26 
27 
28 
29 
30 



-1.92 
-3.71 
-2.19 
-.09 
-. 67 

.66 
.45 
-.94 
.95 

-1.60 

-3.22 
-2.88 
-2.72 
-2.65 
-2 . 28 

-2. 20 
-1.08 

2.02 
54 

-.42 

-. 25 
-2.84 
.64 
.81- 

-.07 

-1.10 
2.34 
1,43 
1.65 
..21 



-1.43 
-2.40" 
-2.87 
.03 
-. 34 

1. 05 
.94 
-1.78 

1.39 
-1.12 

-2.27 
-2.09 
-1.94 
-2.00 
-1.83 

-1.96 
-1.16 
1.86 
-. 25 
-. 20 

-.12 
-3.41 
1.27 
1.13 
.14 

-1.39 
3.04 
2.75 
1.46 

.48 



1. 00 
..77 
.77 
1.72 
1.15 

1.12 
1.19 
.24 
.67 
1.06 

1.31 
1.20 
1.23 
1.05 
.91 

.75 

.48 
.78 
1.23 
1.92 

1.71 
.50 
1.51 
1.12 
.94 

.39 
.96 
.42 
.87 

1.18 




'I 

ON 
I 



63 



61 



. Test 
Item 



31 
32 
33 
34 
35 

36 
37 

3a 

39 
40 

41 
42 
43 
44 
45 

46 
47 
48 
49 
50 

51 
52 
53 
54 
55 

56 
57 
58 
59 
60 



Table 3.3.2 (continued) 



NAEP Math Item Resporise Model Parameter Estimates 
(13 Year\01ds; 1977-78) 

\ ■ 



1;P 
b 



Booklet No. 1 



3-D 



-1.18 
-. 34 
.36 

-3 . 25 
-. 75 



-4 



,88 
.16 
.66 
.13 

, 73 



.84 
1.46 
.68 
.47 
.46 
\ 



.11 
.21 
.11 
.11 
.11 



\ BooklGt No. 2 

■ \ 

1-p \ 3-p 

b \ b ' a 



1.44 


3.75 


2 


00 


.21 


.6.3 


.97 




69 


° .11 


-1.41 


-.88 


1. 


37 


.11 


-.93 


-1.24 




35- 


. 11 


-.71 


-. 41 


1. 


05 


.11 



.87 
-. 75 
-2.93 
-1.90 
1.29 

-.39 
-.75 

.68 
, .pi 

.20 



:. ,78 
-1\03 
-2.4^3' 
-1.81 
1 . 30\ 



1.10 
.42 

1.06 
.78 
.72 



-.31 \ .79 
-.44 'a. 30 



.76 
.08 
.40 



1\.07 
.\84 
1.07 



.03 
.04 
.04 
.04 
.00 

.04 
.10 
.07 
.04 
.10 



1.12 


1 


00 


1 


.11 


.05 


-2 


.01 
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. '■ Table 3..3.2 (continued)- - • ,' ' ' . . ' • 

NAEP Math Item Response Model Parameter Estimates ' . 

•' ■ (llYear Olds 1977-78) / . ' .' , ■ 



Test 
Item 

I- 


1-p 
b 


Booklet No. 1 

'■'3-p' 
. b a • 


c 


b 


Booklet No . 2 

3-p . 

b ' " a 


C 


1~P 

b 


Bookl e t No 
b 


. 3 
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, a 


' c 
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■ - 




-.62 


...-^ :-.'62' ' .64 
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.11 
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1.14 
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1.05 


.00 
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-.82 


.60 


.11 
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3.4 Checking Model Assumptions 

Checking on two model assumptions, unidimensionality and equal' item 
discrimination indices, with respect to the NAEP math booklets, was carried 
out. The results will be presented next. It was not necessary to check 
the level of test speededness because t.fest items were administered ohe at 
a time to examinees and they were given sufficient time on each one to 
provide answers. 1 

Unidimensionali ty ^ . \^ 

Checks on the urfidimensionality of the math booklets were carried out 
with NAEP Math Booklet No. 1 for 13 year olds. A study of the eigenvalue^ 
was "carried out with Black and White samples of 330 examinees. The samples 
were drawn at random from the availabNle pool of examinees taking the math 
booklet^' The largest eigenvalues for each sample are presented in order 
in Table 3.4.1. Comparable results from the Black and White samples 
were obtained: About 15% of the total variance was accounted for by the 
first factor or component and the ratio of the^ first to the second eigen- 
value was (approximately) 2.8 ^2.7 in the Black sample and 2.9 in the ' 

f 

Wliite sample)^ These statistics do not meet Reckase's (1979) .minimal ^ 

{ - 

criteria for unidimensionality. However, since his criteria are arbitrary 
and other goodness of fit evidence would be a,vailable, the decision made 
was to move on to other types of analyses. 

Equal Item Discrimination Indices 

^ ' ~ ' ~ ^' 

Table 3.4.2 provides item difficulty and discrimination (biserial 

correlations) information . for NAEP Math Booklet No. 1 for 13 Year Olds. 

The> information is reported for six groups: Two Black and White samples 

^Somewhat better results were obta:cned from an analysis of the total 
sample (N=^2422) . About l7.6% of the variance was accounted for"by the first 
factor and the ratio' of the first to the second eigenvalue was *3.6. 



Table 3.4.1 



Listing of the Largest Eigenvalues 
for NAEP Math Booklet No. 1 
(13 year olds, 1977-78) 




Table 3.4.2 

. Suminary of Item Statistics' for NAEP Math Booklet No., 1 
(13 Year Olds, 1977-78) 









Item Difficulty Level 




Item 


Discrimination 


Index 




Item 














... . 


















Group 










-Group 








D±acK X 




/ rilacK 
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Table 3.4.2 (continued) 
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Table. 3.4.2 (continued)- 
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White 
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.06 
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.32 
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.03 
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.49 
.58 


.33 
.30 
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56 
57 
58 
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.85 
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.48 
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of 165 examinees, and combined Black and combined White samples of 330 
examineeis/ The results for the four smaller samples are reported here 
although they were of more importance for an analysis reported later. 
The discrimination indices obtained with the Black and Wliite samples re- 
veal two things: The item discrimination indices for the two samples are 
comparable (average absolute difference = ,11; Black values were higher 
22 times; White values were higher^ for 35 items) although the White 
values tended to be a little higher; and more importantly, there is 
substantial variation among the item discrimination indices (.04 to .89 
in the Black sample and -,02 to ,83 in the White sample). 

In a more complete analysis the following results were obtained; 

Item Discrimination 





Booklet 


Sample 
Siz^ 


Tes t 
Length 


Mean 


Indices ^ 

SD 


Booklet 


No, 


1, 9 Year Olds 


2495 


65 


.565 


.260 


Booklet 


No. 


2, 9 Year Olds 


2463 


75 


.565 


.260 


Booklet 


No. 


1, 13 Year Olds 


2500 


58 


^ .585 


.250 


Booklet 


No. 


2, 13 Year Olds 


2433 


62 


.615 


.252 



The results above show clearly that the assumption of equal item discrimination 
indices is violated to a considerable degree I This finding is not surprising 
because item statistics plaiy only a small part in NAEP mathematics test 
developm'ent . It would be ' reasonable , therefore, to expect a- wider range of 
values than might be found on a standardized achievement or aptitude test 
where items with low discrimination indices are most likely , deleted . 
Therefore, it is reasonable to suspect that the two- or three-parameter 



k'or relations were transformed via Fisher ^s Z transformation prior 
to calculating the descriptive statistics.^ The mean is reported on the 
correlation scale. The standard deviation is reported on the scale. 



V ■ 



-65- 



logistic models will provide a '^ore adequate fit to the test results. 
This point. is addressed in more detail in section 3.6. 

3.5 Checking Model Features 

When an item response model fits a test data set, at le^st to an 
adequate degree, two advantages or features are obtained: (1) item 
parameter -^^stimates do not^epend upon the samples of examinees drawn from 
the population of examinees for whom the test is designed (i.e., item \. 
parameter invariance) and (2) expected values of ability estimates do 
not depend upon the choice of test items. The extent to which the first 
feature was obtained with NAEP math data will be presented next. 

Item Parameter Invariance 

The invariance of item difficulty estimates for Whites and Blacks 
with the one-parameter model was investigated' initially, with Math Booklet 
No. 1 for 13 Year Olds. Three hundred and thirty Black examinees were 
located on the NAEP data tape. All these examinees were used in the 
analysis. An equal number of White students were selected at random 
from the same data tape. Next, the Black and the White student samples 
were divided at random into two halves so that four equal-sized (N=165) 
groups of students could be obtained. These groups were labelletf*^"mi 
1/' ^Hs^ite 2," ^'Black 1," and "Black 2. A one-^pa rameter analysis was 
carried out with each groap. ' The plots of "b" values in the two White' 
and Black samples are shown in Figures 3.5.1 and 3.5.2, The plots show 
high relationships between the sets of b values (r = .98). What varia- 
tion there is in- the plots is due to model-data misfit and examinee 
sampling errors. The plots provide a basis for investigating hypotheses 
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Figure 3.5,i Plot o£ b values for the one- parameter model obtained from_ 
two equiyalent white student samples (N=165). 
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Figure 3..5,..2' Plot of b values for the one-parametfer model obtained ftom 
\ tiVo equivalent black student samples (N=165) . 
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concerning the invariance of item parameter estimates' .If'the feature of » 
item invariance is present, -similar plots should be obtained when the I 
Black and White item parameter estimates are compared. Figure 3.5.3 re-^ 
veals clearly that item difficulty estimates differ substantially in the 
first Black and Wliite samples (r = .74). Figure- 3.5.4 provides a repli- 
cation of the Black-White comparison of item difficulty estimates. The 



plot of b values in^Tgure 3.5.4 is very similar' to the plot in Figure 3.5.3 
and both plots differ substantially from the baseline plots shown in 
Figure '3.5.1 and 5.2. 

Figure 3.5.5 provides a plot , of the differences in item difficulty 
estimates between the two I^7hi^e and the two Black samples (r = .06). The 
item parameter estimates obtained in each racial group should estimate 
the same -item parameter value if the feature of item invarian^^^s obtained 
(although the value may be differept in the two racial groups).- Therefore*, 
the expected differences should be zero and the correlation of these differ- 
ences across the set of test items in these two racial groups^ should also 
be zero. In fact, the correlation is very close to zero. the feature . 

\ • ' - 

Of item-ihvariance is present it sKould. exist for any pairings of the data, 
figure 3.5.6 shows that the correlation between b value differences ii> 
the first and second Black and White samples is not zero -'(sLn fact, r = .72!). 
Clearly, item* difficulty estimates obtained with the one-parameter model are 
not invariant -.in the Black and White examinee samples. 

The appropriate conclusion seems to be that item ii^variance ^across the 
two racial groups .is hot obtained. However, we stop short here of a^tribudLng 



the problem to race bias in the test items^ The're are at least two-othe 
plausible explanations: (1) the problem is due to a variable which is 
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F±gu:fe 3.5,3 Plot of b values for the one-parameter model ol?tained from 
the first white and black student'" samples (N=165) . 
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confouxided with race (e.g., achievement scores/ability level - Blacks did 

JO ' 

perform substantially lower on the Math Booklets than Whites - see Table 
3.4.2); and (i2) failure to consider other important item statistics such 
as discrimination (a) and pseudo-chance level (c) . With respect to (2), 
in other words, the problem is due to model-data misfit. But whatever the 
explanation it is clear tha,t the feature of item parameter invariance is 
not obtained.^ 

In a follow-up investigation with NAEP Math Booklet No. 1 with 13 Year 
Olds, the examinee pool was split'^into high and low performers (the cut-off 
point was set at the median) , Each group contained in excess of 1200 
examinees. A one-parameter analysis was carried out with each group. Table 
3.5.1 provides three difficulty estimates for each item: £otal group, 
low ability group, and high ability group. Plots of the three possible 
combinations are presented in Figures 3.5*7, 3.5.8, and 3.5.9'. 

The plots are not 'directly comparable with the earlier ones for race 
because the sample sizes in this analysis are considerably larger. But, 
again it seems clear- that item parameter invariance is not obtained. This 
time, however, item parameter' invariance was not obtained across high and 
low ability scorers. Table 3, 5. -2 and Figures 3.5.10, 3.5.11 and 3.5.12 
provide a similar analysis foJ^'Math Booklet No. 2 with 13 Year Olds and ^ 
again the conclusion is the same. 

At least two criticisms can be made of the previous analyses summarized 
in Tables 3.5.1 and 3.5.2: (1) there is no baseline data available for 

0 

interpreting the plots, and (2) no attempt is made to account for variation 
in items due to their discriminating powet and pseudo-chance level. The 
analysis described next with Math Booklet No. 1 with 13 Year Olds was 
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^Unfortunately the same analyses could not be carried out with, the three- 
' parameter model because of the very small sample sizes. An alternate method- 
^ ology to handle the small sajnples was recently proposed by Linn and Harnisch 

^ (1981). 
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Table 3.5 
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One-Parameter Model Difficulty Estimates 


for 


■ Total, 


Low , 






and High Ability Groups for NAEP Math Booklet No 
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(13 Year Olds, 


1977-78) 
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Group 
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Total 
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Table 3.5.1 (continued) 
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' Abilty Group 
Total. Low 


High 
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1 7Q 
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Figure 3.5/7 Plot of one-parameter model item difficulty estimates for 

the low and high scoring ability, groups on NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). / ' * 
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Figure 3.5.8 



Plot of one-parameter model item difficulty estimates for 
the low and total ability groups on NAEP Math . Booklet 
No. 1 (13 Year Olds, 1977-78). " 
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Figure 3.5.9 Plot of one-parameter model item difficulty estimates for 
the high and total ability groups on NAEP Math Booklet 
No. 1 (13 Year Qlds, 1977-78), 
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Table 3.5.2 



One-Parameter Model Difficulty Estimates for Total,' Low, 
and High. Ability Groups for^NAEP Mafh Booklet No. 2 
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Ability Group 






Item 


Total 


Low 


-Hi- eh— 


• — • 
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■ — - - ' 

.21 


.21 


.28 ' 
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.72 


.91 


.63 




■ ' 3 


-.22 


-.42 


.14 
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■ 6 
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' 2.58 
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, - • .57 
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-3.26 


-2. ,99 - 
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-1.58 


-1.36 


-2.91 




10 
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• -1.97 
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. ' 1.00 
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Table 3 . 5 . 2 (continuedy 
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. . * Ability Group 



Item 




Total 


I • •■• 
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41 




-1.86 
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43' " ■ 
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Figure 3.5.10 Plot of one-parameter model item difficulty estimates 'for 

the low and high scoring ability groups \on NAEP Math Booklet 
No. -2 (13 Year- oUs, 1977-78). 
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Figure 3.5.11 Plot of one-parameter model item difficulty estimates^ for 
the -low.^nd total ability groups on NAEP Math Booklet 
No. 12 '(13 Year olds, 1977-78). 



/ 



ERIC 



9'J 



s 



5 



3. 50 -T 



2. 50- 



1. 50- 



50- 



50^ 



'I. 50- 



^2.50^ X 



X. 



X 



x^ 





x 



X 



X 



X X 
X XX 

#^ 

.X 
X 



fx 



V/ 



-3,50- .X ^ —^Jrj ' - |- 



-3. 50 



-2, 50 -1. 50 



50 



50 



1.50 2.50 



HMOLE SPMPLE 



3. 50 



^Figure 



3.5.12 Plot of one-parameter model' item difficulty estimates for 
the high and total ability groups on NAEP Math Booklet 
No. 2^fT?^ar Olds., 1977-78). 



carried out to address the two deficiencies. A group of 2400 examinees was 

found with the 1200 lowest ability students and 1200 hig?iest ability,,.. . 

students. The (approximately) 22 middle abijlity^^stnidents were deleted * 
ftom the analysis. Next, thgu24'00 examinees v/ere divided on a random 
basis into two,..eq'U:ai sub-groups of 1200 examinees. Each sub-group was 
used to obtain one-parameter and three-parameter model item estim^ates. 
Figures 3.5.13 and 3.5.14 provide the plots of b values in the two samples 
obtained wit,h the one- and three-parameter logistic ""mcdels . The item 
parameter estimates in the two samples with either test model are nearly 
identical. Thus, item parameter invariance across random groups is^ estab- 
lished. Next, the 2400 examinees were divided intio two equal-sized low 
and high ability groups (again, N=1200) and the analyses and plots carried 
out with the random groups were repeated. The results for the one- and 
three-parameter models are reported in Figures 3.5.15 and 3.5.16 respec- 
tively. , 

IF the feature of item invariance was present all fo-lir plots should have 
looked the same. In fact, the plots in Figures 3.5.15 and 3 . 5 . 16~~aYB— s-ub.- 
stantinlly different from those in Figure 3.5.13 and 3.5.14. However, it is 
iK^t plansihle nt this time to explain the ditferencGS in terms of a failure 
to account for essential item statistics (i.e., discrimination and pseudo- 
level) since the one-parameter and threerparameter plots of item difficulties 
for high and low ability examinees shown in Figures 3.5.15 and 3.5.16 are 
similar. One possible explanation which remains is that item parameter esti- 
mation is not done very well when extreme gVoups are used.-^ Of course 
another possibility is that the test items are functioning differently in 
the two ability groups, i.e., item parameters are not invariant across 
ability groups. 

^The close fit between the lihree-parameter model and several data set:s 
reported in section 3.6 suggest that this explanation is highly plausible. 
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Figure 3.5.13. Plot of one-parameter' model item difficulty estimates 
obtained in two equivalent samples with NAEP >fath Booklet No. 1 
(13 Year Olds, 1977-78, N=1200) . 
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Figure 3.3.14, Plot of three-parameter model item difficulty estimates 
obtained in two equivalent samples wifch NAEP Math Booklet No. 1 
(13 Year_01ds^. _1.977-78j_. N=12^ 
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Figur(% 3. 5 . 15 . Plot of one-pararaeter model item difficulty estimates 
obtained in'low and high ability groups with NAEP Math Booklet 
No. 1(13 Year Olds, 1977-78, N?^12D0) . , 
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Figure 3.5.16. Plot of three-parameter raoJel item difficulty estimates 
obtained in low and high ability groups with NAEP Math Booklet 
No. l (13 Year Olds, 1977-78, N=1200) . ■• ' 
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Tables 3.5.3 and 3.5.4 provide the results frpm a different type of 
analysis but one which --^.eems promising for addressing item parameter in- 
^^ariance. It has become common practice to address item invar iance by 
conducting a statistical analysis qf the b value differences between two 
groups of examinees (for example, Blacks and Whites). The problem is, 
as was..stated in section 2. 1,*^ when the sample size is large, even prac- 
tically insignificant differences are often statistically significant . The 

method described next depends upon replication and practically significant 

<i ■ ■ * 

differences. In- this analy'sts, a practically significant differelnce of 

interest (referred to as the critical value) was ;3elected (=.50) and two 

equal-sized Black and l#iite examinee samples w(fre divided into two eq-ual- 

si^ed sub-samples (N=165) . A one-parameter analysis of Math Booklet No. 1 

with 13 Year^ Olds was carried out for each group. In Table 3.5.3 items 

with b value differeuces exceeding .50 between either the first Black and 

White samples, or the secolid Black and \^ite " samples , are shown. Twenty- 

seven of 58 items excee:ded the critical value with the first samples; 25 

of the 58 items exceeded th§' critical value with the second samples, Ifo 

the differences vjere'Mue to chance factors only, 20.0% of the test items 

would be predicted to be identified in both samples. In fact, 59.2% of the 

27 Items (1.6 Items) identified as different in the fxrsr samples 

were identified as different in the second samples. "^^ihen viewed m 

the other direction, 16 of the 25 items (or 80%) identified in the second 

samples were also identified in the first samples. 

Table 3 . 5 . 4 provides the results of a replication of the study wi.th 

Ma'th Booklet No. 2 for 13 Year Olds. Eighteen items were identified in 

sample 1; 15 items were identified in s-ample 2; 13 of the items were common 

In other words, of the 18 items identified in the first samples, 72.2% were 



102 



' -90- 

Table 3. .5. 3 . ' . 

Item Difficulty Differences in Black-White Samples 
for NAEP Math Booklet Nq. 1^ 
(13: year olds, 1977-78) 



Item 



Black and White' Samples (N=165) 
1 ' '2 



4 
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— X . 


UD 






/ 










ft ~ ■ 
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- .92 
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1 .08 
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. -7 O 
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'^.74 


23. 
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— 1 






. O J 


24 
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- .84 


26 
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28 
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33 
















- .64 


35 
















- .63 


36 










-1 


.83 




-2.15 


38 












.86 




.71 


39 












.66 




■ " - .80 


40 












.51 




.83 


43 












.57 






47, 










-2 


.03 




-1.08 


48 
















- .70 


51 












.59 




- .82 


52 












;65 






53. 
















- .51 


55 












.71 




.63 


56 












.62 






57 












.86 




1.53 


58 










c 


.70 







^Otily items with a difference >|.50| in one or both samples are reported. 
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Table 3.5.4 



Item Difficultly Differences in Black-UHiite Samples 
for NAEP Math Booklet No. 2^ 
(13 year olds, 1977-78)^ 



Black and White Samples (N=220) 



4 




■ - .57 


6 


' .51 




9 ■ 


1.11 


.88 


10 


1.45 


.66 


11 


.98 


.88 








12 


.86 




13 


.70 




15 


.53 




19 




. ' .62 


20 


1.27 




22 


-2.03 


0 -1.95 


26 


- .75 


-1:36 


29- 


- .58 


.49 


30 


.54 


.83 


35 


1.03 


.76 


50 


.62 


.62 


51 


-1.33 


-1.18 


53 


.63 


.94 


57 


-1.84 


-1.54 


61 


- .84 . 


- .65 



4£> 



^Only items with a "difference ^|.50| in one or both samples are reported 



.1 04 
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identified in the second samples; and of the 15 itemi? identified .in the 
second Samples, 86.7% of the items were identified in the first samples. . 
Tf clinnc.o Factors only wore operating, about 7% of Che Items would be expected 
to be coiiihionly identified, in the two samples (15/62 x 18/62): Clearly, it can- 
not be argued that iteni invariance across the two groups is present when the 
b values are estimated usin^ the one-parameter logistic test model. 

The method described above seems like a promising approach for address- 
in):, the problem of item invariance. And^'^rom one point of vi^w, it really 
doesn't matter what the causes o.f the differences are. The fact is that 
item invarian'ce is not found across a variable that can be used to describe 
the examinee population. It would be misleading therefore to^ offer only 
a single set of item statistics. \^hile not investigated here, commonly 
identified items c^n be additionally studied ^to attempt to detect the source 
of the problem(s). At this stage, directions of any observed differences 
can be also investigated. ^ . 
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o . . ^ 

3.6 Checking Additional Model Predictions ' ' -* ^ t r 

This, section of our worl^c is divided inta two parts: Residual Analyses, 
and Research Hypothesis Investig-ations . 



Residual -Analyses 

To carry' out residual analyses with Math Booklets. Nos . 1, 2; and 3 for 

■ ^- I 

9 and 13 Year Olds it was necessary to prepare a computer. program. A . 
listing and sample output of our program- is presented in Appendix B. ■ The 
program was prepared to be ^.compatible with the item and ability parameter 
estimation output from LOGIST. The program 'provides both residuals and 
standardized residuals for each test -item at various ability levels (the 
number is selected by the user) . (Twelve ability levels were chosen in 
our investigation.) In addition, fit statistics are avaiflable for each 
test item (found by summing over ability levels) , fot each ability level 
(found bv. summing over test items), and for the total test (found by summing 
ovor ability levels and test* items). 



A sample set of standardized residuals for Math\Booklet . 1 with 
13 Year Olds^ obtained with the one-parameter model arfe shown in Figures 
3.6.1 to 3.6.11. Two features of the plots in the figi^res (and other plots 
we studied) are the cyclic patterns and the large size of the standardized 
residuals.^ I tem-. patterns like those in Figures ^ 3 . 6 . 1 , 3.6.3, 3.6.4, 3:6.5, 
and 3:6.10 were obtained for items with relatively high biserial correla- 
tions. Item patterns like those in Figures 3.6.6, 3.6.7, 3.6.8, and 3.6.9 
were obtained for items with relatively low biserial correlations. Also, 
the standardized residuals tended to be high.. In Table 3.6.1 it can bp 
sfHMi Ihnt (approKLmntolv) 25% of the standardized residual^s exceeded a 
va^Iue of 3 when the one-paraifieter mod'el was fit to the test data. This , 

"^Standardized residual plots for items 1 c^:) 10, and 36 .are shown in 
the figures. » ■ . 
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Figure 3.6.1 Standardized residual jplot obtained ^ith the one-parameter 
model for test item 1 from NAEP Math Booklet No. 1 
(13 Year Olds, 1977-78). 
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.Figure 3.6.2 Standardized residual plot obtained with the one-parameter 
" " ' model f6r test item -2 from N4-CrMath Booklet No. 1 
^ , , (13' Year Olds, 1977-78). 
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Figure 3.6.3 ^Standardized residual plot obtained with the one-parameter 
model for test item 3- from NAEP Math Booklet No. 1 
(13 Year olds,M977-78). 
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Figure 3.6.4 Standardized residual plot obtained with the one-parame 
model for test item 4 from NAEP Math Booklet No. 1 
(^13 Year Olds, 1977-78). 
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Figure 3,6,5 Standardized residual plot obtained with the one-parameter 
model for test item 5 from NAEP ^^ath Booklet No, 1 
(13 Year Olds, 1977-78), 
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Figure 3.6.6 Standardized residual plot obtained with the one-parameter 
model for test item 6 from NAEP Math Booklet, No. 1 
(13 Year Olds, 1977-78).. 
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Figure 3.6.7 Standardized residual plot obtained with the one-parameter 
model for test item 7 fxom NAEP Math Booklet No . 1 . 
^ (13 Year Olds, 1977-78). I ^ 
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Figure 3.6.8 



Standardized residual plot obtained with the one-parameter 
model for test item 8 from NAEP Math Booklet No. 1 
(13 Year Olds, 1977-78). 
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Figure 3.6.9 Standardized residual plot obtained with the one-parameter 
mo.del for test item 9 from NAEP Math Booklet No. 1 
(13 Year Olds, 1977-78) .\ ' 
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Figure 3 6 10 Standardized residual plot obtained with the one-parame 
• model for test Item 10 from NAEP Math Booklet No. 1 

(13 Year Olds, 1977-78) . 
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Figure 3.6.11 Standardized residual plot obtained with the on^'-parameter 
model for test item 36 from NAEP,Math Booklet No. 1 
(13 Year Olds, 1977-78). . ■ 
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result was obtained with 6 tes t - booklets . If the model data fit had been 
good, the distribution of standardized residuals would have been approxi- 
mately normal. ^ 

The standardized re'sidual plots obtained from fitting the three- 
parameter model and shown 'in Figures 3 .6 .12 to 3 . 6 . 22 reveal dramatically 
different patterns. The cyclic patterns which were so evident in the 
first olovon fir,iires are Rone,"and the sizes of the standardized residuals 
are substantially smaller., • 

fagie 3.6.1 p'rov rd irg^-^cnmpl'g1re7~g^ ^ — ■ — - 

Standardized residuals obtained with the one- and three-parameter models 
for six Math ^Booklets. In all cases the standardized residuals are con-L 
siderably smaller with the three-parameter model and the' dis tributions are 
approximately normal. '. ^ ' 

Table 3,6.2 reports the average raw ^^^nd absolute-valued standardized 
resjiduals^ at 12 abili.ty levels with the one- and three-parameter models for the 
'sam(^ six Math Booklets.. A^ain, the results in this table reveal the 
sup(^riority of the three-parameter model. Also, it is clear that the 

three-parameter model is especially effective at low levels of al^ility. 
/ » _ 



' -Research Hypothesis Investigations 

<? ' 

The residual analysis results in the last section were most inter- 
esting but it seemed -desirable to investigate the misfit statistics 
further.' Tables 3.6.3 to 3 .6 . 6 ^provide the basic information we worked 
with for f out of the Math Booklets. 



^The average raw standardized residuals provide information about the 
size and dire'ption" of the misfit between the observed results and the ICCs . 
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Figure 3. 6 . 12 .Standardized residual plot obtained with the three- 
parameter model for tes.t; item 1 from NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). 



ERLC 



119 



4.0- 



S.0- 



2.0- 



1.0- 



0.0 



-i.oo- 



-2.0- 



-3.0- 



-5. O- 

-9, 



X 



X 



X X 5^ 



X 



X 



-2. O 



^ 1 

-1.0 



,0.0 



f 

I.O 



1^- X. 



T 



Figure 3,6,13 Standardized residual plot obtained with the three- 
parameter model for test item 2 from NAEP Math Booklet 
No, 1 (13 Year/Olds, 1977-78), 
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Figure 'i.6^i4~ Standardized residual plot obtained with the three- 
parameter model for test Item 3 from NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). 
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No. 1 (13 Year Olds, 1977-78). 
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Fieure 3.6.16 Standardized residual plot obtained v(ith the three- 
.Figure J.o p^^^^^,^^ ^„„del for test item 5 f rom Wp M^th Booklet 
No. 1 (13 Year OLds, 1977-78'). 
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Figure 3.6.17 Standardized residual plot obtained with the' three- 
parameter model fot: test item- 6 from NAEP Math Booklet 
No.. 1 (13 Year Olds, 1977-78). t 
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Figure 3.6.18 Standardized residual plot obtained with the three- 
parameter model for test item 7 from NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). 
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Figure 3.6.19 



Standardi2ed residual plot obtained with the three- 
parameter- model for test item 8 from NAEP Math Bo6klet 
Nol 1 (13 Year Olds, 1977-78). 
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Figure 3.6. 


20 


Standardized residual plot obtained with the three- 
parameter model for test Item 9 from NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). 
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Figure 3.-6. StaBda-rd-lzecLresidual plot obtained with the three- ' 
Figure p3^3^eter model for test item 10 from NAEP Math Booklet 

No. 1 (13 Year Olds, 1977-78). 
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Figure 3.6.22 Standardized residual plot obtained with the three- 
parameter model for test Item 36 from NAEP Math Booklet 
No. 1 (13 Year Olds, 1977-78). 



Table 3.6.1 



Analysis of Standardized Residuals witn the One- 
and Three-Parametjer Logistic' Models iox Six 1977-78 NAEP 
— Mathematics Booklets 



NAEP 
Booklet 


Logistic 
\ Model 


10 to 1 


Percent of 
1 1 1 to 2 


Residuals . 
|2 to 3| 


|over 3| 


Booklet 1 




35. 9 


21.5 


17.3 


25.3 


(9 Year Oias; 




66 7 


2A. A 


. 6.7 


2.3 


Booklet 2 


1 


37. 1 


25.3 


13.8 


23 . 8 


\y Year Uius; 




67 A 


2A . 7 


5.7 


2.2 


Booklet 3 


1 




23. A 


15. a" 


21.1 


(9 Year Olds) 


3 


64.0 


24.8 


8.0 


- 3.3 


Booklet 1 


■ 1 


AO. 7 


22 . 1 


16.5 


20.7 


(-13 Year Olds) 


3 


65. A 


25. 1 


7.8 


1.7 


Booklet 2 


1 


A2.6 


2A.2 


16.3 


16.9 


(13 Year Olds) 


3 


67.2 


26. 1 


5.7 


1 . 1 


Booklet 3 


1 


3A.3 


2A.5 


17.5 


23.7 


(13 Year Olds) 


3 


61 .0 


26.8 


8.2 


A.O 

■ i 



^At the 9 Year Old Level, there were 780 standardized residuals 
(65 test items x 12 ability levels). At the 13 Year Old Level, there 
were 690 standardized residuals (58 test items x,12 ability levels). 
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Table 3.6.2 



Analysis of Standardized Residuals at Twelve Ablllcy Levels with the One- 
and Three-Paramecer Logistic ModL'ls for Six 1977-7B yAEP Machen-^atlcs Booklets 



KAEt' 
SooV.let 



Test, 
Length 



Logistic Sample 
Statistic Model Size -2.75 



^2.25 -1.75 



Ability Level 
-1.25 -.75 -.25 .25 



.75 1,25 1,75 2,25 2 



Total 

75 (unweighted) 



-1 

olds) 


65 




1 
3 


249^ 
2495 




27 
29 


43 
50 


111 
108 


220 
212 


, 331 
333 


4 

4 85 
454 


446 
470 


395 
403 


276 
• 271 


122 
100 


21 
21 


8 
9 






Average 
Residual 


1 
3 






77 

00 


.99 
.24 


.89 
.27 


.79 
.12 


.37 
.16 


.20 
.04 


.14 
.08 


-.28 
-.18 


-.26 
-.48 


-.39 
-.36 


-.11 
-.32 


-.10 
-.16 






Average 

Absolute 

Residual 


1 
3 




1 


75 ' 
81 


2.40 
.90 


2.82 
1.02 


3.35 
.74 


2.35 
l.QO 


1.80 
.94 


1.62 

.62 


2,35 

.87 


2.64 

' .99 


2.40 
.85 


1.19 
.91 


.i35 
.88 


2 

olds) 


75 




1 
3 


• 2463 
2463 




10 

23 


46 . 
64 


116 

89 


234 
218 


■334 
346 


437 
417 


474 
497 


397 
403 


272 
230 


87 
107 


39 
34 


7 
6 






" Average 
Residual 


1 
3 






60 

;16— 


.74 
--.14- 


.58 
-.02 


.7.1_ 
.34 


„.28 
.50 


— .01" 
.19 


V02^ 
-.03 


"-^.14 
-.18 


-.23 


*^ .08 
-.05 


-.05 
-.16 


.02 
.01 






Aver-age 

Absolute 

Residual 


1 
3 




1 


.55 
..84 


2.42 
.95 


3.02 
.83 


3.10 
1.05 


2.28 
1.D2 


1.49 
.94 


1.59 
1.04 


2.36 
.89 


.2.75 
1.01 


1.71 
.87 


1.31 
.90 


.53 



.25 
.05 



2.13 
.88 



.23 
.01 



2.03 
,90 



, • ... 
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Table 3.6.2 (continued) 



NAEP 
Booklet 



Test 

Length 



Booklet 3 

(9 year olds) 



68 



Lop.lstlc Sample 

Statistic 



Average 

Kcsi uual 

Aver/ige 

Absolute 

Residual 



-2.75 -2.25 -1.75 -1.25 - 



Ability 
.75 -.25 



Level 
.25 .75 



1.25 1.75* 2.25 2.75 



Total 
(unweighted) 



1 

3 


2438 
2438 


29 
28 


44 
62 . 


120 
108 


174 
177 


326 
283 


410 
4 38 


533 
319 


446 
410 


186 
219 


89 
88 


44 
, 29 


9 
9 


1 
3 




.84 
.36 


.57 
.36 


.60 
-.02 


.17 
-.22 


-.01 
-.55 


-.04 
-.50 


.04 
.01 


.30 
.37 


.09 
.39 


-:o9 

.34 


-.22 
.09 


-,03 
.09 



2.20 
• .94 



2.13 
.91 



3.04 
1.01 



2.69 
.97 



2.07 1.25 
1.03 .90 



2.06 2.61 
.97 .95 



1.97 1.49 
.90 1.18 



1.23 
.95 



.95 
.64 



.19 

.,06 



1,57 
.95 



Booklet 1 

(L3 yctir olds) 



58 



Average 



Average 

Absolute 

ResiduaX 



1 


2422 


14 


54 


91 


224 


3 


2422 


24 


50 


^lA^- 


1-94- 


— 1— 




.67 


.88 


.66 


.33 


3 




-.02 


.08 


.06 


-.07 



1.76 
1.27 



2.59 
1.02 



2.74 
.97 



325 503.- 
'3X8r^ ^40 



.03 
.07 



.25 
.27 



2.64^ 
.84 



2.20 1.63 
.76 .92 



_467- 


-339 ' 


-*2T5 To2 


44 


3 


509 


368 


248 


90 


32 


11 


.60 


.17 


-.12 


.07 


,11 


-.09 


-.20 


-.22 


-.59 


-.03 


-,12 


-.43 


1.68 


2.08 


2.06 


1.62 


1.30 


,67 


.•79 


.84 


1.07 


.84 


.86 


.99 



.28 
-.10 



1.92 
.93 



ERJC 



133 



134 
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* ' Table 3.6.2 Xcontinued) 



NAEP 


Ttiit 




Logistic 


Sample 












Ability 


Level 












Total 




booklet' 


Length 


Statistic 


Model 


^ Size^ . 


-2 


75 


-2.25 


-1.75 


-1.25 


-.75 


-.25 


.25 


.75 


1.25 


1.75 


2125 


2.75 


(unweighted) 




Booklet 2 


62 






2A33 




20 


39 


118 


241 


308 


447 


463 


'392 


2/»0 


i2l 


22 


10 






(13 year olds) 








2A33- 




15 


45 


121 


2 30 


334 


429 


440 


402 


259 


94, 


26 


11 










Average'^ 




































0 




' Residual , 








90 


.92 


1.06 


.67 


.14 




-.01 


-.05 


-.27 


-.13 


-.03 


-.01 


.26 
















03 


.17 


.13 


.22 


.03 


.10 


-.08 


-,.24 


-.26 


-.27 


-.12 


-.13 


-.03 








Average 






















^» 


















Absolute 


































M 






Residua,l 






2 


08 


1.90 


2.98 


2.69 


1.64 


1 . 7.9 


1.54 


1.95 


2.10 


1.64 


1.00 


.73 




ts> 






i 








76 


.79 


.87 


.82 


.81 


1.05 


.94 


.85 


. 85 


.9^ 


.69 


.70 


. .84 • ^ 

\ ■ 


O 
1 


Booklet 3 


73 






2A69" . 




12 


38 


96 


215 " 


. 400 


540 


403 


341 


237 


120 


51 


8 






(13 year olds) 




f~ 




2A69 




2A ' 


6 53 


106" 


203 . 


328 


462 


■ 499 


393 


212 


106 


38 


6 










Average 
































.29 








Residual 








A5 


.74 


.78 


.56 


.26 


.31 


.23 


-.16 


-.01 


.15 


-.01 


.10 
















49 


.08 


.04 


.27 


-.23 


.01 


-.15 


-.15 


-.34 


-.31 


-.36 • 


-.13 


-.03 








Average 








































Absolute 








































Residual 




\ 


1 


50 


2.42 


2.97 


3.34 


2.78 


2.01 


1.57 


2.11 


2.47 


*2,32 


1.97 


.84 


2.19 














1 


25 


• .94 


1.29 


1.05 . 


.8-7 


1.03 


.97 


.84 


1.04 


1.06 


" .99 


.68 


1.00 
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Table 3.6.3 



NAE? Hath Booklet No. 1 
RAfiir I t civA Statistical and Cl a^slflcat o r y Inf o rmation 
^ ^ (9 year olds, 1977-78) 



Tes t 
Uepv 


S f" A n'd ^1 i*"'d i 7 p d 

■ 1-p 


Residua lb""'" 
.3-p 


I tern 
Dlf f iculty^ 


TtCMn 

3 

Discrimination"^ 


Con ten t 
Categorv''* 


5 

Forma t 


— ^ — 


■ 1.27 


0.62 


.55 


.62 


3 


1 


2 


r.73 


0.60 


.47 


.69 


3 


1 




1.27 


0 .85 


.55 


.65 


1 ■ 


1 






9 ' 9 A 


91 


34 • 


2 


1 - 


5 


2.28 


1.57 


.89 


.39 


' 2 


1 


6 


3.'26— 


1.08 


t 

:7o 


.33 


2 


1 


7 


2.00 


0.88 


.12 


.37 


2 


2 


8 


0'.59 


0.82 


.33 


.56 


2 


2 




1 7 '5 ' 
1 . / J 


u . o J 


46 


. 47 


2 


1 


10 


1.53 


0.63 


.39 


.65 


5 


1 


11 


2.18 


O.J 9 


.89 


.77 


4 


2 


12 


2.03 


1.01 


.84 ■ 


.75 


4 


2 


13 


2.45 


0.84 ' 


.88 


.80 


4 


2 


1 k 


Z . J J 


1 7"^ 




' . 76 


4 


2 


15 


2.61 ' 

r 


1.06 


.81 


.80 


4 


2 


16, 


3.05 


2.16 


.75 


.79 


4 


2 


17 


3.20 


1.00 


.46 


.-3-5 


1 


1 


]8 


0.49 • 


0.59 


. 81 


• 59 


1 


2 


19 


0.P.6 


. 1.30° 


.S5 


.51 


2 


1 


20 


0.85 


0..73 


.63 


.63 


4. 


2 


;u 


2.35 


0.48 


.40 


.75 


4 


2 


22 


2 . 2,6 


0.74 


.20 


.60 


5 


1 


23 


1.84 


0.65 


.53 


.62 


1 


1 . 


2-'t 


2.50 


0 .58 


.82 • 


.79 


4 


2 




'V 1.55 


/ 0.86 


. 40 


.68 


4 


2 




1-p i one'-pnrameter logistic model; 3-p 


= three-parameter 


1 ogistlc 


model.' 



'eric 



' '^Trem difficulty i proportion of examinees in thi' NAEP .snriple answering', the 
U^sl ,(toni correctly (N = 2495). ^ 

. Item discrimination E biserial correlation between item and the to tal ^^te-s t 
score. 

■^Content. Categories : 1 - Story Problems, 2 - Geometry, -3 - Definitions, 
4 - Calculations, 5 -Measurement, 6 - Graphs and Figures. 



^•ormat: i - multiple choice, 2 - open response. 



I 
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Table 3.6.3 (continued) 

, \ NAEP Math Booklet No. 1 ^ 
Basic Item Statistical and Classif icatory Information 
(9 year ol ds, .1977_-7_8) _ 



Test 


Standardized 


Residuals-^ 


Item 2 


Item ^ 


Content ^ 


IfTTl f* ^ 

r U L Illd L 


Item 


1-p 


3-p 


Difficulty 


Discrimination 


Category 


26 


2.64 


0.88 


.49 


. 77 


4 


2 


27 


1.85 


0.86 


.68 , 


.71 


4 


2 


28 


1.08 


0.94 


.36^ 


.63 : 


4 


2' ' 


29 


1.41 


0<540 


. 77 


.69 


4 


2 


30 




0.88 


.68 


.78 


4 


2 


31 


1.92 


0.99 


.69 


. 72 


6 


1 


32 


4.48 


1. 33 


.03 


.14 


3 


1 


33 


4.92 


0.69 


"^19 


.14 


3 


1 


34 


1.12^,^^ 


0.92 


.64 


.54 


5 


1 


35 


0.9Z>=^ 


1.13 


.80 


.62 


6 


1 


36 


✓1^41 


1.10 


' .65 


.67 


4 


2 


37 


1.25 


0.56 


.09 


.60 


4 


2 


38 


1.33 


0.84 


.94 


.43 


3 


1 


39 


3.53 


^.72 


. 20 


.26 


1 


1 


AO 


4 . 00 


0.58 


.17 


.22 


4 


1 


'-f X 


2.26 


1.12 


.20 


.73 


1 


2 


42 


0.69 


0. 38 


.17 


.57 


4 


2 


43 


1.22 


0.58 


.02 


.61 


4 


2 




1.10 


1.10 


.01 


.59 


4 


2 


45 


3.55 


0.87 


.29 


.28 


5 


2 


46 


1.72 


0.60 


.36 


.51 


4 


1 


47 


2.63 


1.11 


.54 


.40 


5 


1 


48 


1.18 


0.61 


.83 


-.67 


6 


1 " 


AQ 


2.36 


0.93 


. 29 


. 50 


6 


1 


50 


4.38 


, 0.47 


.66 


.27 


1 


1 


SI 


4.1:8 


0.69 


.25 


.21 


1 


1 


s? 


5.51* ' 


- 0.88 


. 35 


.19 


« 2 


1 


J -J 


3.19 


6.66 


.09 


. 22 


2 


1 


J *-* 


2.67 


0.97 


.09 


.31 


2 


1 


55 


0.58 


0.65 


.01 


.49 


. 6^ 


2 


56 


1.43 


0.68 


.12 


.64 


1 


2 


57 


1.51 


1.16 


.48 


.53 


2* 


1 


58 


1.11 


0.91 


.24 


.53 


2 


1 


.59 


2.32 k 


0.44 


.28 


.48 


2» 


1 


60 


0.99 % 


o^S-^ 


-.21 


.51 


1 


2 


• 61 


1.54 


0.92 


.10 


,53 


"5 


2 


62 


1.46 


1.47 


.85 


.60 ' - 


3 , 


2 


63 


1.53 


1.17 


^ .48 


, .67 


4 


2 


64 


1.16 


, 0.53 . 


.35 


.49 


2 


1 


65 " 


3771 


0.9.4 ' 


" .27' 


..v24 


3 


. 1 



138, 
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Table 3 .,6 •A 



NAEP Math Booklet No. 2 
-Basie--I-tem--Sta-t4s-t4.x-al~and-Classi^ 
(9 year olds, 1977-78) 



Test 


Standardized Residuals^ 


Item 


Item 


Content 


Format 


t^.tem 


1-p 


3-p ^ Difficulty^ Discrimination-" 






1 


3.27 


0.67 


.77 


. 31 


J 


1 


2 




0. 64 


.78 


.31 


-5 


1 

X 


3 


0.73 


0.90 


.92 


.60 


4 


2 


4 


1.50 


0.77 


.87 


.70 


4 


2 


5 


1.38 . 


1 . 27 


.88 r 


.65 


/, 


9 

L 


6 


1.35 


1.22 


.78 


. 67 




9 


7 


± . D / 


0. 96 


.86 


.71 




9 


8 


1.44 


0. 88 


.82 


.70 


A 


2 


9 


2. 39 


J. ■ J. U 


.59 


.76 


4 


2 


e 

10 


2.57 


0 79 


.60 


.76 


/, 


9 
L 


11 


2.87 


\ 

0.65 


.50 


.78 


A 


9 


' 12 


/ . 34 


, 0.79 


.50 


.74 




9 




0.94 


0.59 


.08 


.46 


2 


1 


14 


1.00 


0.83 


.37 ^ 


.58 


1 


1 


15 


1.19 


1.31 


.73 


.57 


6 


1 


16 


1.31 


0.71 


.57 * 


.63 


6 


1 


17 


1.03 


6.77 


.74 


.64 


4 


2 


io 


1.06 


0.73 


.73 


. 65 


4 \ 


2 


19 


1.59 


1.06 


.56 


.68 . 


4 • 


2 


20 


1.31 


0.99 


.14 


.56 


1 


1 


21 


1.77 


0.55 


' .63 


.71 


6 


1 


' 22 


2.L7 


1.01 


.57 


.72 


6 


1 


23 


2.26 


1 . 06 


.39 


. 71 


6 


1 


1 


1..18 


0.67 


f96 


.68 ' 


3 


1 


1 25 


0.83 


0.70 


.96 


.60 


3 


1 




■'"1-p = one-parameter logistic 


model; 3-p = 


three-parameter 


logistic 


model . 




2 

Item difficulty 


= proportion 


of examinees 


in the NAEP s-ample answering the 


test 


item correctly 


(N = 2463) . 











^Ttem discrimination E biserial correlation between item and the total test 



score 
4 



Content Categories: 1 - Story Problems, 2 - Geometry, 3 - Definitions 
4 - Calculations, 5 - Measurement, 6 - Graphs and Figures. 

*^Format: 1 - multiple choice, 2 - open response. 
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Table 3.6.4 (continued) 

NAEP Math Booklet No. 2 
Basic 'item Statistical and Classif icatory Information 
^ ^9 year olds> 1977-78) 



Test 


Standardized 


1 

Residua, s 


Item 


Item 


Content 


Format 5 


I t em 


1-D 


3-P 


Difficulty^ 


Discrimination 


Category^ 


^ 

26 


1.10 


0.69 


.97 


.68 


3 


1 


27 


0.67 


0.69 


• .94 


.52 


3 


1 


28 


0.74 


0.84 


.92 


.56 


3 


1 


29 


4.80 


0.70 


.19 


.18 


5 


1 




9 ft7 


n 77 


. 20 


-.32 


5 


1 


31 


1.03 


0.91 


.25 


.60 


4 


2 


32 


» 1.67 


0.96 


.27 


.66 


1 


2 


33 


1.87 


1.03 


.4^ 


.69 


3 


2 


34 


1.83 


1.09 


.52 


.69 


3 


2 








.47 


.67 


3 


2 


36 


3.16 


0.82 


.39 


.34 


2 


1 


37 , 


0.63 


0.69 


■ .84 


. 60 


2 


1 


38 


1.20 


0.61 


.19 


.47 


2 


1 


39 


4.43" 


1.18 


.25 ■ 


.21 


1 


1 








63 


. 70 


4 


2 


41 


2.29 


0.66 


.40 


.7^ 


4 


2 


42 


2.58 


0.74 


.72 


.78 . • 


4 


2 


43 


2.98 


1.09 


.56 


.81 ^ 


4 


2 


44 


. 2.58 


0.65 


.74 


•79 


4 


2- 


45 


o /. n 
z . 


U . / -J 


Aft 


. 75 


4 


2 


46 


2.44 


0.88 


.19 


.37 


2 


1 


47 


1.51 


0.81 


.90 


.42 


1 


2 


48 


1.09 


0.54 


.75 


.66 


3 


2 


49 


1.11 


1.23 


• 50 


.63 




2 


.' 50 


0.60 


0.75 


.41 


.55 


• 3 


1 


51 


3.39 


Q.83 


.80 


.27 


5 


1 


52 


2.29 


0.76 


.71 


.76 


3 


1 


53 


1.96 


0.45 


.50 


.64 


3 


1 


54 


2.67 


1.43 


.44 


.45 


3 


1 


55 


3.89 


0.64 


.25 


.25 


1 


1 


56 


2.25 


0.89 


.54 


.43 


1 


1 


57 


2.61 


0.52 


. .37 


■ .41 


1 


1 


58 


0.67 


0.56 


.66 


.60 


1 


1 


59 


1.14 


, 0.80 


.50 


.61 


1 


1 


60 


1.40 , 

1 


1.25 


.23 


.52 


4 


1 
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Table 3.6.4 (continued) 

NAE? Math Booklet No. 2 
-^a^i-c— I^ em S tatisti c al and-Gjra-s-s^-^-i-^-t^€hry--fe-f^ 
(9 year olds,- 1977-78) 



Test 
Item 



f 1 

.Standardized Residuals Item 

Dif f iculty^ 



1-p 



3-p 



1 1 em' 
Discrimination^ 



Content 
Category^ 



Fajmat 



61 


4.08 


5. 


44 


.88 


.13 


2 


1 


62 ' 


3.07 


0. 


73 


.44 


.35 


2 


1 


63 


4.76 


0. 


56 


.21 


.16 


3 


1 


64 


5.88 


0. 


84 


.14 


.06 


3 


1 


65 


4.63 


0. 


54 


.25 


.19 


3 


1 


66 


0.81 


0. 


66 


.12 


.45 


4 


2 


67 


1.68 


1 


78 


.26 


.50 


1 


2 


68 


0.82 


0 


48 


.01 


.54 


2 


2 


69 


2.15 


1 


05 


.49 


.42 


1 


■1 


70 


2.63 


0 


94 


.08 


.22 


1 


2 


71 


1.65 


0 


67 


.06 ■ . 


• 35 


2 


2 


7.2 


■ 1.21 


; 0 


.63 


.04 


..58 - 


4 


2 


73 


1.76 


0 


.83 


.34 , 


.44 


5 


2 


74 


0.59 


0 


.99 


.39 


.57 


6 


2 


75 


2.66 


0 


.74 


.34 


.35^ 


5 


1 
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\ 

V 

\ 

\ 



Table 3.6.5 



NAEP Math Booklet No, 1 
Basic Item Statistical and Classif icatory Information 
: (n ye'ar olds, IQ77-7R^ 





i eb L 


Standardized Residuals"*" 


1 1 em 


I tem 


Content 


Format^ 




Item 


1-P . 


3-p 


Difficulty^ 


Discrimipation-" 


Category 






1 
1 


1.47 


.84 


. 85 


. 70 


1 


2 




9 ' 


.68 


.44 


. 93 


.61 


3 


1 




.o 


.71 


.85 


.95 


.62 


3 


1 






1 1 


^ 1 . y ^ 




.81 


5 


2 




5 


1 .74 


i89 


.65 


.72. 


4 


1 




D 


1 .80 


.96 


. 3'6 


.48 


2 


1 




7 


1 . 70 


.64 


.40 


.49 


2 


1 




?l 
o 


3.80 


' 1 .47 


. 70 


. 29 


2 


1 




0 


2 . .1 J 






. 43 


. ^ 1 . 


1 




io 


1 .59 


.64 


.81 


.72 


• 5 


1 




1 t 


1 .47 


.86 


95 


.75 


4 


2 




1 0 


1 .47 


1.31° 




. 74 


i 


2 . 






1.61 


1 . 1 1 




.75 


. -^4 


2 




1 A 


1.21 


m 




. 70 


4 


2 


• 


15 - 


'.97 , 


.88 


.89 


.66 • 


4 


2 




16 


-~ . 1.11 


1 .39 


.88 


.58 ' 


4 


2 




17 


1 .86 


.98 


- .73 


.47 


5 


1 




18 


.96 


.83 


. 14 ^ 


. .54 


1 


2 




1 Q 


. ■ 2.42 


I .42 


.^2 


■ ' .75 


4 


2 


# 


20 


3.30 


.42 


.59 


.84 


4 


2 




21 


3.08 


.53 


.56 


.82 


4 


2 




22 


.68 


.48 


.93 


.46 


3 


1 




23 


2.85 


■ .71 


.36 


.38 


3 


1 




24 


1 .88 


.89 


.33 


.48 


3 


1 




•• 25 


1.15 


.98 


.52 


.64 


1 


2 






M-p'"= one-parameter logistic 


model; 3-p = 


three-parameter logistic model. 








^Item difficulty = 


proportion 


of examinees 


in the NAEP sample 


answering the 




• 


test 


item correctly (N 


- 2500) . 














•^Item discrimination H biserial correlation 


between item and 


the total 





test score. 



6 = 



'eric 



^^Content Categories: 

Story Problems, 2 = Geometry, 3 = Definitions, 4 = Calculations, 5 = M^easurement , 
Graphs and Figures. 

^Format: . ' 

multiple-choice, 2 = open response. 



U2 
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Table 3.6.5 (continued) 

NAEP Math Booklet No.. 1 
Basic Item Statistical and Classif icatory Information 
■ (13 year olds, 1977-78) 



Test 


Standardized 


Residuals-'- " ' 


Item 


Item 


Content 


Format^ 


item 


1-p 


-> P 


"n-i -F-Fi r-iil t-ir^ 
U i. L L i I- U 1. L y 




Cat esorv'^ 


• 


26 


• 2.32 


.46 


.73 


.41 


2 


1 


27 


•1.06* 


.81 


.10 


.51 


2 


1 


28 


4.62 


.77 


.22 


.18 


2 


2 


29 


.92 


.77. 


.18 


.■57. 


5 


2 


30 


1.9/ 


Q 
. OJ 




■ DU 


] 


1 


31 


.80 • 


.73 


.74. 


.64 


2 


1 


32 


2.06 


1 . 56 


.58 


.64 


1 


1 


33 


1.13 


.64 


.42 


.49 


1 


1 




.75 


.56 


.96 


.46 


2 


■ 1 


35 


2.36 


1 Q "7 
1.0/ 


• DO 




2 


1 


36 


7.08 


1.19 


.21 


-.01 


1 


1 


37 


1 .36 


.66 " 


.37 


.47 


. 2 


1 


38 


2.63 


.67 


.78 


.80 


3 


1 


39 


3.37 


.73 


.70 


.36 


_3 


1 


40 


1 "7 O 
1 . // 


Q ^ 

. op 


. P D 


70 


1 


1 


41 


1.16 


.96 


.27 


.62 


3 


1 


42 


.60 


.93 


.69 


.60 


2 


1 . 


43 


.87 


.81 


.78 


.60 


2 


1 


44 


1 .58 


1 .93 


.68 


.59 


4 


2 


45 


1 . 1 0 


1 A9 
1 . 0 Z 


45 


. 64 


4 


2 


46 


2.01 ' 


.90 


.34 


.63 


1 


/ 1 


47 


4. 63 


.98 


. 1 1 


, . 10 


2 


1 


48 


1 .69 


1.11 


. 15 


.48 


3 


1 


49 


1 . 20 ■ 


.83 


. 49 


. 54 


it 


? 


50 


. 77 


.80 


.84 


.62 


1 


1 


51 


3.30 


.57 


■""".18 


.27 


1 


1 


52 


5.03 


.96 


.60 


.26 


1 


1 


53 


1 .37 


.31 


.82 


.45 




1 . 

2 


54 


1 .19 


1.19 


.73 


■ .63 . 




55 


1 .83 


.83 


.25 


.68 


6 


2 


56 


.49 


.74 


.72 


.59 


1 


1 


57 


2.48 


• 95 


.31 


.73 


5 


2 


58 


.83 


.71 


.74 


.62" 


4 


2 
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Table 3,6.6 



NAEP Math Booklet No. 2 
Basic Item Statistical and Classif Icatory Inf orTiiaLion 
(13 year olds, 1977-78) ■ 



Test, 
[tern 


Standardized Residualgl 
1-P - 3-p 


X tem 
Dif f iculty2 


Tteni 
Discr inma t i on 


Content 
Cci tcgory ^ 


Format^ 


1 


1.01 


1.06 


.58 ■ 


.60 


4 


2 


2 


1.13 


0.85 


.48 


.67 


4- 


2 


3 


2.39 


1.74 


.65 


.53 


4 


2 




1.92 


0.72 


.69 


.50 


1 


1 


#5 ■ 


1.49 


0.86 


.57 


.69 


3 


1 


5 


0.87 


1.03 


.18 


.55 


2 


2 


7 


1.00 


1.15 


.51 


.63 


5 


2 


8 


0.56 


6.53 


.96 


.58 


1 


2 


9 


2.25 


0.52 


.85 


.84 


4 . 


2 


10 


2*33 


0.62 


.84 


.84 


4" 


2 " 


11 


2.20 


1.31 


.82 


.84 


4 


2 


12 


2.11 


0.56 


.79 


.82 


4 


2 


13 


0.93 


0.67 


.92 


.68 


2 


1 


14 


,2 • 17 


0.92 


.42 


.48 


4 


1 


15 


1.20 


1.02 


. 30 


.61 


2 


1 ' 


16 


0.71 


0.61 


. uV 


. DO 


A 


9 
L 


17 


0. 79 


- 0.55 


.85 


.69 


4 


1 


18 , 


0.93 


0.51 


. 86 


.70 


4 


2 


19 


1.00 


0.77. 


.95 


•50 


4 


2 


20 


0.99 


0.94 


• 95 


.68 


4- 


2 . 


21 


1.13 


0.76 


.95 


.56 


4 


2 


22 


6.. 17 


1.14 


.06 


-.07 


2 


1 


23 


1.77 


0.66 


.38 


.74 


^ 4 


2 


24 


1.57 


0.71 


.45 


.74 


, 4 


2 


25 " 


1.12 


1.20 


• 61 . ; 


• 63, 


. 4 


2 




1-p = one 


V 

-parameter logis 


cic model ; 3-p 


H three-pa rame.ter 


logistic 


model . 



Item difficulty = proporcion of examinees in the NAE? sair.ple answering the 
(■est item correctly C'^ = 2433). 

■^Item discrimination = hiserial correlation between item and the total test 
score. 

^Content Categories: 1 - Story Problems, 2 - Geometry, 3 - Definitions, , 
4 - Calculations, 5 - Measurement, 6 - Graphs and Figures. 

"^Pormatr. 1 - multiple choice, 2''- open response. 
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Table 3.6.6 (continued) 

NAEP Math Booklet No . 2 
Basic Item Statistical and Classif icatory Information 
(13 year olds, 1977-78) 



Item 


Standardized Residuals-'- 
' l~p 3-p 


Item 
Dlfflcul|y^2 


Item 
Discrimination^ 


Content 
Category^ 


Forma 


26 


3.45 


1.00 


.88 


.24 


• 2 


1 


27 


3.63 


0.89 


.55 


.36 


2 


1 


28 


3.24 


1.48 


.24 


.49 


1 


2 


29 , 


0.62 


0.90 


.91 


.59 


1 ' 


1 


30 


1.07 


1.25 


.16 


.54 


1 


2 


31 


1.54 


0.67 


.30 


.67 


3 


1 


32 


3.03 


0.99 


.67 


.44 


, 3 


1 


33 


' 1.05 


0. 33 


.95 


.77 


3 > 


1 


34 


0.74 • 


0.62 ■ 


.86 


.65 


1 


1 


35 


1.02 


1.16 


.22 


■ .57 


1 


2 


36 


0.74 


0 . 55 


.59 


.64 


6 " 


1 


37 


2.20 


0.65 


.67 


.77 


6 


1 


38 


1.53 


0.70 


-34 


.61 


6 


1 


39 


0.62 


0.60 


.50 


.64 


4 


2 


40- 


. 1.46 


0.76 


.45 


.64 


1 


1 


41 


0.85 


0.70 


.88 


.69 


A 


2 


42 


1.80 


1.69 


.78 


.73 


4 


2 


43 


0.81 >, 


0. 79 


.78 


.59 


1 


1 


"44 


3.61 


0.80 


.73 


.37 


2 


1 


45 


^1.64 


0.76 


.66 


.53 


1 


1 


46 


1.08 


0.77 


.■81 


.68 


1 


1 


47 


1.36 


0. B2 


.80 


.76 


1 


1 


48 


1.24 . 


0.84 - 


.26 


-.65 


3 


2 


49 


1.83 


0.36 


.17 


.68 


3 


2 


50 


1.51 


1.06 


.63 


.72 


3 


2 


5l 


6.21 


■ 1.28 


.32 


.17 


2 


1 


52 


2.99 


0.65 


' .17 


.32 


2 


1 


53 


2. 13 


0.51 


.38 


.75 


1 


2 


54 


1.23 


0.69 


.86 


.55 


2 


■ 1 


55 


1.05 


0.53 


•47 


.56 


2 


1 


56 


2.41 


0.89 


.50 


.80 


1 


2 


57 


6.38 


0.76 


.13 


.10 


1 


1 


58 


2.53 


0.78 ^ 


.17 


.56. 


6 


1 


59 


3.57 


1.19 


.19 


.45'' ' 


6 


1 


60 


1.12 


0.64 


.75 ■ 


.56 


6 


1 


61 


.1.06 


0.92 


.64 


.58 


■ 4 


2 


62 


1.71 


0.83 


.29 


i .70 


4 


2 



"eric 
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Our initial preliminary studies are reported in Figures 3.6.23 to 

3 . 6— 2 -8 . — - Figure 3 — 6v-2jh:s-hew s— t-h e-^el-a-t-iotrsh^^^ -b^ wee n - u n e - p a r a iiie L et ^ 

model residuals and classical itein difficulties. The -outstanding 
features are the large size of the residuals and the tendeacy for" the ' 
most difficult items to have the highest residuals. Possibly this 

o 

latter problem is due to the guessing behavior of examinees. In a similar 
plot with three-parameter model residuals shown in Figure 3.6.24, the 
standardized 'residuals are substantially smaller and it appears that by^ 
estimating item pseudo-chance level parameters*, the tendency for the 
hij'Jiest residuals to be obtained with the most difficult items is reduced". 

Figure 3.6.25 provides a plot of one-parameter model standardized 
residuals and classical item discrimination indices for four of the. Math 
Booklets combined'. A strong curvilinear relationship is evident.^ Items 
with relatively high or low classical discrimination indices have the 
highest standardized residuals .<» Figure 3. 6 . 26, provides the same plot 
using three-parameter model standardized residuals. The curvilinear 
relationship disappears-. Substantially better fits are obtained when 
variations in discriminating powers of test items are handled in the 
chosen model . * - - 

Figures 3.6.27 and 3.6.28 provide comparable information to. the 
previous two figures except that the latter two figures use the informa- 
tion f rom ^ single test booklet. The trends in the results are identical. 



\ 

\ 



Figure 3.6. 23\ Scatterplot of one-parameter standardized residuals and item 
dif ficulti'^^s for 9 and 13 Year Old Math Booklets Nos. 1 and 2. 
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Figure 3.6.2^*. Scatterplot of three-parameter standardized residuals and item 
difficulties for 9 -and 13 Year Gld Math Booklets Nos. 1 and 2. 
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lire 3 . 6 . 25 Scattcrplot of onc-paranctor standardized rosidualc and iL'om discrinlnatipn^ 
^. ''indices for 9 arid 13 Year Old Math Booklets Nos. 1 ^and 2. 
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3.6.2t. Scattorplot of onc-paraTnctc'r otandardizod residuals and item dlscriinination 
indices for 9 Year Old Math' BooklGt No. 2 .c 
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attorplot of thrcc-paranctor standardized residuals and item discrtoi- 
ndiaes for 9 Year Old Math Booklet No. 2 . ' 
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TH'ese initial analyses were encouraging because they provided several 
insights into possible reasons for item misfit. Next, a more comprehensive 
analysis o|| the test items was initiated. Seven different analyses were 
carried ou| on four of the test booklets. In Addition, the analyses were 
carried out on a ct)mbined set of Math Booklets. 

Math Booklet • Tables 

No. 1, 9 Year Olds 3 . 6 . 7 to 3 . 6 . 13 

No. 2,/9 Year Olds 3.6.14 to 3.6.20 

No. 1, 13 Year Olds 3.6.21 to 3.6.27 

No. 2, 13 Year Olds 3 . 6 . 28 to 3 . 6 . 34 

Combined 3.6.35 to 3.6.41 

By combining booklets and obtaining more test items it. was possible to more 
clearly study the trends in the results. 

Since the trends in all o.f the analyses at the Math Booklet level are the 
same, only the results for the combined Math Booklets will be discussed further 

Table 3.6.35 

• Intercorrelations among five key variables. 

1. There is a high negative correlation (rF~.61) between one-parameter stand- 
ardized residuals and classical item discrimination indices.-^ The result 
suggests that the poorest fitting items al^e the least discriminating. 
Perhaps this is due, in part, to examinee Wiessing behavior. 

2. The most difficult test items are the least dXs criminating (r=.41). 
Again, perhaps the result is due to examinee glassing on hard test items. 

3. There is a substantial correlation (r=.49) betweenNltem format and 
classical item discrimination indices. Open-ended t^^t items tend 
to have higher discrimination indices than do multiple-^2.hoic^e items. 
Again, it is noted that guessing is a factor in multiple-bhoice test 
performance but plays almost no part with open-ended test i r^s . 



The higher one-parameter model residuals are 'associated with, the multiple- 
choice test items; the lower one-parameter model residuals are assoc^^a^ted 
with the open-ended items. "^x 



^This correlation is misleading because the actual relationship between 
the two variables is non-linear. 
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Table 3.6.7 



Correlations Among Several NAEP Math Item Variables 
(Booklet No. 1, 65 Items, 9 Year Olds,., 1977-78) 



Variable 



Item Order 

S candardized 
Residual (1-p) 

S taruiardi zed . 
Kusidunl (3-p) 

iieni Di L'f iculLy (p) 

Item Discrimination (r) 

Format (F) 



SR(l-p) SR(3-p) 



.05 



-.19 



.16 



.46 -.32 



.11 -.60 



,35 



.03 
.43 



-.06 

-.34 

.02 
.02 
.57 



l=Mul tip le~ Choice ; 2=0pen-Ended . 



■erjc 



154 
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Table 3.6.8 ' 

Association Between Standardized Residuals 
and NAEP Item Conten,t 'Classif icntions 
(Booklet No. 1, 65 Items', 9 Year' Olds , 1977-78) 



Content 
Category 



Number 
of 

items % 



Standardized Residuals 
1-p 3-p 

SR(<1.0) SR(>1.0) SR(<1.0) SR(>1.0) 

(n= 8 ) (n= 57 ) (n= 48 ) (n= 17 ) 



Storv Problems 10 



20.0 



80.0 



90.0 



10.0 



Geometry 



14 



14.3 



85.7 



64.3 



35,7 



Definitions 



Calculations 



23 



0.0 



8.7 



100.0 



91.3 



7-1.4 
69.6 



28.6 



30.4 



Measu remen t 



0.0 



100:0 



83:3 



16.7 



Graphs and 
Figures 



40.0 



60.0 



80.0 



20.0 



X^= 6.25 
d.£.= 5 P= .282 



X - 2.63 
d.'f . = 5 



P= .757 



'ERIC 



155 
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. Table 3.6.9 



Association Between S tandardiy.ed Residuals 
and Item Formats 
(Booklet No. 1, 65 Items, 9 Year Olds, 1977-78) 




Fonna t 



Multiple-Choi'ce 



Open-Ended 



.Standardized 
Residuals 



SR(<1.0) 
SR(>1.0) 

SR(<1.0) 
SR(>1.0) 



1-p Results 
N % 



2 

32 

6 
25 



3.1 
49.2 

9.2 
38.5 



>• = 1.62 
d.f.= 1 p= .203 



3-p Results 



156 



Table 3.6.10 



Association Between Standardized Residuals 
and Item Difficulties 





(Booklet No. 1, 65 Items, 


9 Year Olds, 


1977-78) 








Difficulty 


Standardized 




Resul ts 


3-p 


Results 
% 




.Level 


Residuals 


N - 




N 




Hard (p<.5) 


SR(<1.0) 


4 


. 6.2 


32 


49. 


2 


> 


SR(>1.0) 


■ 33 


50.8 


5 


. 7. 


7 


Easy (p>.5) 


SR(<1.0) 


4 


6.2 


16 


24,. 


6 




SR(>1.0) 


24 


36.9 . 


12 


18. 


5 








102 


2 


5.66 








d . f . = 


1 P=.967 


d . f . = 


1 p= 





I 



\ 




Difficulty 

Level 



-142- 



Table 3.6.11, 



Association Between Item Formats \ 

' and Item Difficulties \ > 

(Booklet No. 1,-65 It>ems, 9 Year Olds, 1977-78) 



' I'o rma t 



Hard (p<.5) 



Mcil ciple-Choice 
i 

Open-Ended 



20. 

17 



3018 

2 6 J, 2 
.1 



Easy (p>.5) 



Multiple-Choice 
Open-Ended 



14 
14 



21.5 
21.5 



X'= .005 
d.f.= 1 p= .9-42 



158 
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Table 3.6.12 

DescripLive Statistical, Analysis of 
Standardized Residuals 
(Booklet No. 1, 65 Items,. 9 Year Olds, 1977-78) 



Dif f icul cy 
Level 



t o rmn i: 



• "NuiTiber 
^ ■ of 
Items 



1-p RcvSults 
X • SD 



3~p Results 

X ,^ SD 



iijrd (jx.5) MulCipIo-Choice 20 

^ OpcMi-FMulod 17 



2.84 1,31 
1.55 .79 



.78 
.80 



.23 
.23 



Easy (p>.5) Multiple-Choice 14 

Open-Etided 14 



1.98 
1.95 



1.09 
.74 



1.03 
1.01 



.46 
.48 



/ 



150 
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Table 3.6.13 



Relationship Between • 1 tern Discrimination Indices 

.and Standardized Residuals 
(Booklet Njx.__.1, .65 Items., 9Jeax„.01ds, 197 7-78) 



— — 3 

Standardized 
Model Residuals 


-cOl to .30 


Discrimination 
.31 .to .50 


Indice.'S ■ 
.51 to .70 


.71 to 1.00 




CIO)' 


. ( 13 ) 


( 29 ) 


- ( 13 ) 


1-p 0.00 to 1.00 


0.0 


7.7 


24.1 


0.0 ; 


1.01 to 2.00 


no 


■^0 ft 


■ 79 4 


15 4 


over 2.00 


inn n 


U -L . 








2 

Y LO OA 

Eta-. 743 


d.f.^ 6 


p= .010 




3-p 0.00 to l.OCg 


90.0 


69.2 


f. ■ 

75.9 


61.5 


l.OJ to 2.00 


10,0 


23.1 


24.1 


■ 30 '8 


over- 2 . 00 


0.0 


7.7 


0.0 


7 . 6 






d.f.= 6 


P=.575 






Eta=,231 









Number of test items appear in brackets. 
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... Table 3.6.14 

Correlations Among Several NAEP Math Item Variables 
(Booklet No. 2, 75 Items, 9 year Olds, 1977-78) \ 



Variable 



SR(l-p) SR(3-p) p r\ 



Jl=Multiple-Choicc; ' 2=0pen-Ended . 



Item Order .19 ' .08 -.51 " -.44 ^ ■ -.09 

Standardized. 

Residual (1-p) M7 -.26 -.60 \--27 
Standardized 

Residual (3-p) . • -15 -.19 -.01 

Item Difficulty (p) ^ -^0 .6^ 

Item Discrimination (r) p ' "^9 

Format (F) * . 



161 
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0 



) 

Assoc la tioiji 
and NAEP 
(Booklet No. 



'Table 3.6.15 

Between Standardized Residuals 
Item Content Classifications 
2, 75 Items, 9 Year Olds, .1977-78) 



\ 



Content 
Catego ry 



Number 

of 
Itenus 



Standardized Residuals ' 
! 1-p ' ■ ' 3-p- \. 

SR(<1.0) SR(>1.0) SR(<1.0) SR(>1.0) 

(n4 12 ) (n= 63 ) (n= 57 ) (n= 18 ) 



Story Problems 13 



13.4 



84.6 



76.9 



23.1 



(laometrv 



33.3 



66.7 



88.9 



11.1 



Definitions 



19 



21. 1 



78.9 



73.7 



26.3 



Calculations 



23 



8.7 



91.3 



73.9 



26.1 



Measurement 



0.0 



100.0 



100.0 



0.0 



Graphs and 
F i Kur (.•••; 6 



16.7 



83 .3 



50.0 



50.0 



4.24 



d. f .= 5 _ P= .515 



;/•= 4.7-4 
d. f .= 5 P= :449- 
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Table 3.6.16 



Association Bhtween Standardized Residuals 
and Item Fomats • 
(Booklet No. 2, 75 Items, 9 Year Olds, 1977-78) 





Format 


Residuals - 


x~p 

. K 


!\e suits >j 

. % 


o 

J- 

N 


p Results 
% 




Mul L iole— Cho i ri^ 




■ 

i 

Q 
O 


./ 

in 7 


J/ 


■ 

AO "7 


• 


\ 


■ SR(>1.0) 


32 


4.2.7 


■8 


10.7 




Open-Ended 


ft 

SR(<1.0) 


4 


"'5.3 


25 


.33.3 




0 


SR(>1.0) 


31 




10 


. 13.3 




J 






.482 . • 


9 


= .358 








■d.f ,= 


1 'V=.487 


d.f. 


= 1 ■ p= . 5-5; 

















TRIG 



16\3 



Table 3.6.17 ' 

Association Between Standardized Residual's 
and Item Difficulties 
(Booklet 'No'. 2, 75 Items, 9 Year Olds, 1977-78) 



Difficulty 
Level 



Standardized 
'Residuals 



1-p Results 

% 



N 



3-p Results 



Hard (p<.5) 



SR(<1.0) 
SR(>1.0) 



6 

34. 



8.0 



45.3 



31' 
9 



41.3 
12.0 



Easy (p>.5) 



SR(<1.0-) 
SR(>1.0)' 



.6 
29 



8.0 
38.7 



0 . 



26 
9 



d.f.= 1 p=1.00 d.f.=l 



^34.7 
12.0 . 

.003 

p= .95-7 



164 
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Table 3.6.18 

Association Between Item Formats 
and Item Difficulties 



• 




(Booklet No. 2, 75 Items, 


9 Year 


Olds, 1977-78) 




Difficulty 
Level 


Format 


N 


% 




• 


'Hard (p<.5) 


Multiple-Choice 


23 


30.7 








Open-En-ded 


17 


22.7 


\ 


• 


Easy (p>.5) 


Mul tiple-Choice 


17 


22.7 




t» 




Open-Ended 


18 


24.0 


• 


• 






d.f .=1 


293 

p=.588 





TRIG 



16, 
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Table 3.6.19 



Descriptive Statistical Analysis of 
Standardized Residuals 
(Booklet No, 2, 75 Items, 9 Year Olds, 1977-78) 







Difficulty 
Level 


Format 


Number. 

of 1 
Items / 


i-p Results / 
X SD / 


3-p 

X , 


Results 


Hard (p<.5)' 


iultiple-Choice 


23^ 


/ 

/ 

2.69 1.47 / 

/ 


.81 


.25 



©pen-Ended 17/ 1.67 .68 / .89 .31 




Table 3.6.20 



Model 



RelaCionship Between Item Discrimination Indices 
and Standardized Residuals 
(Booklet No. 2, 75 Items, 9 Year Olds, 1977--78) 



Standardized 
Residuals 



-.01 to .30 



1--P 0.00 to 1.00 

1.01 to 2.00 
over 2. bo 



( 9 )^ 

0.0 

0.0 
100.0 



X^^ 50.77 
Eta= .744 



( 18 ) 
11.1 
27.8 
61.1 

d.f.= 6 



( 34 ) 
29.4 
70.6 
0.0 

P=.000 



Discr iminacion Indices 

.31 to .30" • .51 to .70 .71 to 1.00 



( 14 ) 
0.0 
14.3 
85.7 



3-p 



0.00 to 1.00 



1.01 to 2.00 



over 2.00 



77.8 
11.1 
11.1 



X^= 8.78 



Eta= .114 



83.3 
16.7 
0.0 

d.f .=6 



73.5 
26.5 
0.0 

P=.186 



71.4 
28 .'6 
0.0 



Number of test items appe.'ir in brackets, 



^ERIC 



167 



V 
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Table 3.6.21 

Correlations Among Several NAEP Math Item Variables 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



Variable 



Item Order 

Stanclardized 
Residual (1-p) 

Standardized 
Residual (3-p) 

Jlvm ])i f r i c'u 1 ly (p) 

J Lem ' Discriniina Lion (r) 

Format (F) 



SR(l-p) SR(3-p) ; 



,06 



.03 



16 



-.27 



.38 



-.03 



•20 

-.57 

.05 
.Al 



-.09 

-.09 

.27 
.08 
.46 



1 l=Multiple-Choice; 2=0pen-ER-ded . 



•eric 
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Table 3.6.22 

Association Between Standardized Residuals^- 
and NAEP Item Cont;ent Classifications 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78)' 



Standardized Residuals 
Number 1-p 3-p- 

of SR(<1.0) SR(>1.0) • SR(<1.0) SR(>1.0) 

Items (n= 13) (n= 45) (n= 45) (n= 13) 



Content 
Category 



Story Problems 14 



21.4 



78.6 



85.7 



14.3 



Ceome L ry 



14 



28.6 



71.4 



85.7 



14.3 



Def ini lIdus 



33.3 



66.7 



88.9 



11.1 



Calcula tions 



15 



13.3 



86.7 



53.3 



46.7 



Measurement 



20.0 



80.0 



80.0 



20.0 



Graphs and 
i"' igu reH 



0.0 



100.0 



100.0 



0.0 



.>*= 1.95 
d. f .= 5 p= .856 



=7.10 

-a . f . = 5 



p= .213 



ERIC 



163 
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Table 3.6.23 

Association Between Standardized Residuals 
and Item Formats 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



Format 


Standardized 


1- 


'P Results 


3-P 

iM -. 


Results 

/a 


Mul tiple- Choice 


SR(<1.0) 


9 


15.5 


31 


53.4 




SR(>1.0) 


27 


46.6 


. 5 


8.6 


Open-Ended 


SR(<i .0) 


4 


6.9 


14 


24.1 




. SR(>1.0) 


18 


■ 31.0 


8 


13.8 








= .078 


. 

A - 


2.78 






d.f . 


= 1 p=.780 


d. f .= 


1 p=.09 



> 

•f ■. 



17 0 
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Dif f iculty 
Level . 



Table 3.6.24 ' 

AflsociuLioa Between Standardized Residuals 
and Item Difficulties 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



S tandardized 
Residuals 



1-p .Results 
N % 



3-p Results 
N % 



Hard (p<.5) 



SR(<1.0) 
SR(>1.0) 



2 

20 



3. A 
3A.5 



19 
3 



32.8 
5.2 



Easy (p>.5) 



SR(<1.0) 
SR(>1.0) 



11 19.0 26 

25 A3.1 10 

X'=2.A9 

d.f.=l p=.115 d.f.=l 



X^=.862 



A4.8 
17.2 

p=.3^ 
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Table 3.6.25 

Association Between Item Formats' ^• 
aud Item Difficulties 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



Difficulty 
Level 


Format 


N 


9 

z 


Hard (p<.5) 


Multiple-Choice 


15 


.25.9 




DpL-n-Hnded 


7 


12.1 


Easy (p>.5) 


Multiple- Choice 


21 


36.2 




Opcn-Ended 


15 








X^=.22 








. d.f.=l 


p=.638 



172 
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Table 3.6.26 



Descriptive Statistical Analysis of 
Standardized Residuals 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



Difficulty 
Liive L 



■■i>rnin l 



Number 

of 
1 Cems 



l-i) Results 



SI) 



3-p Results 
X SI) 



Hard (p<.5) 



Multiple-Choice 15 
Open-Ended 7 



2.38 
1 .88 



1.60 
1.33 



.84 
.94 



.19 
.31 



Easy (p>.5) Multiple-Choice 21 

Onen-iindcd . 15 



1.72 
1.73 



1.21 
.83 



.84 
1. 09 



.38 
,45 



173 
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£ ... 



Model 



Table 3.6.27 

Relacionsh Ip Between Item Discriniinat ion Indices 

and Standardized Residuals 
(Booklet No. 1, 58 Items, 13 Year Olds, 1977-78) 



SLaudardi ::<.»d 
Res i duals . 



1-p 0.00 to 1.00 

I. 01 uo 2.00 
over 2.00' 



Discr imination indices 
01 to .30 .31 .to. .bO .51. to .70 



of' 

0.0 
100.0 

'^^= 26.9 
Eta= .628 



( 15 ) 
13.3' 
53.3 
33.3 

d . f . = 



( 26 ) 
A2.3 
50.0 
7,.7 ' 

p= . 000 



.71 to 1.00 



( 11 ) 
0.0 
A5.5 
5A.5 



;3-p 



0.00 to 1.00 
1.01 to 2.00 
Mver 2.00 



66.7 
33.3 
0.0 

>r=2.5i 

Eta_=.208 

&1: 



86.7 
13.3 
0.0 
d.f .=3 



.80.8 
19.2 
0.0 



,474 



,63.6 
36.4' 
■ 0.0 



'Number of test iCe:n.s appc-ir in brackets. 



J 



'ERIC 
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Table 3.6.2", . 

CorrelaLiony, Among^ Several NAEP Math Item Variables 
(BookletiNo. 2, 62 I^s,^13 Year Olds, 1977-78) 



Variable 



SR(l-p) SR(3-p) 



Item Order 

Standardized 
Residual (1-p) ■ 

Standardized 
Residual (3-p) 

iLu'Ti Difficulty (p) 

Item Discrimination (r) 

Format (F) 



,21 



.13 



,29 



-.29 



.43 



24 



-.16 

-.71 

-.29 
.-3g 



.29 

•.31 

.17 
.07 

'.44 



^ l=MultiplG- Choice ; ■"2-Open-Ended , 



TRIG 



175 



Conc4nt 
Category 



Table- 3.6.29 

^ssocifi Lion- Between Standardized Re.s tdun l^s, , ''^ 
and NAEP Ltera Content Classifications 
(Bo-olclSt No. 2,' 62- Items, 13 Year Olds, 1977-78) 



Standardized Residuals 
Number ' 1-p 3-?" . 

of SR(<1.P) ■ SR(>1.0) SR(<1.0) SR(.>l.-0) 

Items ■ (n= 15 ) (n=' 47 ) ', (n= 47 ) (n= 15 ) 



Story Rroblems 15 



26.7 



73.3 



-80.0 



20.0 



Geometrs 



11 



18.2 



81.8 



63.6 



36.4 



Definitions 

Calculations 
\ 

Measuremen t\ 



22 



0.0 



100. D' 



100.0 



31. S ^ '68.2 



0.0 .. 



85.7 14.3 



7-7.3 



0.0 



22.7 



100.0 



Graplis and 
]• Lgures 



16.7 



83.3 



83.3 



:16.7 - 



6.52 



4.75 
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Table' ^.6.30' 



Associntid'n Be.tween Standardized Residuals 
■ and ft&m Formats 
(Booklet ^o. 2; 62 Items, 13 Year Olds, 1977-78) 



Forma c 



Mu 1 1 ip ie- Cho i ce 



Standardized 
Residuals 



SR(<1.0) 
SR(>1.0) 



1-p Results 
N Z 



5 
25 



8.1 
40.3 




3-p Result^ 
4> 7% 



41.9 
6.5 



Open-Ended 



3R(<1.0^ 
. SR(>1,0) 



10 16.1 

22 ^^-a^-rS—' 

>'^=1.09 
d.f.= l ■ 'p=.297 



21 33.9 
"11 3,7 . 7 

X^=2.67 
d.f.---l p=.102 



V 



177 
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Table 3.6. 31 



J 



Association Between Standardized Residuals 
' ' .-arid Item Difficulties 

(Booklet No.^ 1\ 62 Items, 13 Year Olds igyy-'TS) 



Dif f j.Gulty 
Level 



Standardized 
Residuals 



1-p Results. 

% 



N 



3--p Results 
N % 





/ 

% 






S 1— £i : 








Hard 5) 




SR(£-i 


0) 




3.2 


1=7 


27.4 ■ 


/ 




• SR(>1 


0) 


23 


37.1 


8 


12.9 


Easy (p>.^5) 
















: / 


SR(<1 


0) 


13 


21.0 


. 30 


48.4 






SR(>1. 


0) 


■ 24 


38.7- 


7 


11.3 ^' 










■ X^=4.60 




X^=.77 












d.f.-=i 


p= .032 


d.f .=1 


p=.''38 

/ 






* 













i?8 



• 






/ f 

1 ■ ' • 


, / 










• 


1 
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1 
t 
1 

1 


Table 3.6.32 


! 








• 




1 

Association Between Item Formats 
anp Item Difficulties 
(Booklet No^ 2, 62 Items, 13 Year Olds, 1977-7'S) 












Difficulty 
Level 


Format 


K % 










• 


Hard (p<.5) 


Multiple- 


/ 

Choice 12 19.4 














Open-Ended 13 21.0 






•4 






Easy (p>.5) 


Multiple- 


Choice . "18 ,29.0 








o 






Open-Ended ' 19 ^ 30.6 










• 






0 

d.f 1 p=1.00 










• 


















- 














V 




1 


ft 




• 


















• 




•eric 

.i ■ 




? 

0 


4. 











-164- 



Table 3.6.33 



Dif f icuity 
Level 



Des(iript:ive Statistical Analysis of 
\ Standardized -Residuals 
(Booklet Nq. 2, 62 Items', 13 Year oUs , 1977-\78) 



ormaE 



Number 

of 
I terns 




3-p Results 
X SD 



Hc7rd (p<.5) 

/ 



Multiple-Choice 12 
Open-Euded 13 



3.0,7 

I 

1.^9 



2. 06 
.72 



Easy (p>.5) .. Multiple-i-Choice 18 
Open-Ended ' ° 19 



1.71 
1.^6 



1.04 
..62 




.24 
.31 



.16 

,38 



X: 
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Table 3.6.34* 

Relationship Between Item Discrimination Indices^ 

and Standardized Re.siducils 
(Booklet No. 2, 62 Itfems, 13 Year Olds, 1977-78) 



Model 



i-p 



Standardized 
Residuals 



0.00 to 1.00 
1.01 to 2.00 
over 2.00 



-.01 to .30 



( 4 
0.0 
0.0 

100.0 

34.40 
Eta= -666 



Discrimination Indices 
.31 to .50 .51 to .70 



( ,..9 ) 
11.1 
11.1 
77.8 

d.f .= 



( 36 ) 
' 38.6 
55.6 
5.6 



,000 



.71 to 1.00 



( 13 ) 
0.0 
46.2 
53.8 



3-p 



0 . 00 to 1 . 00 
1.01 to 2.00 
over 2.00 



50.0 
50.0 
0.0 

1.56 
Eta= -158 



77.8 
22.2 
0.0 
d.f .= 



77 .8 
22.2 
0.0 

p= .669 



76.9 
23.1 
0.0 



'Number of test items appear in brackets, 



'ERJC 



181 
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Table 3.6.35 . 

CorrelaLiuiib Among Several NAEP Math Item Variables 
(Booklet Nq.i1 and 2, 260 Items, 9 and 13 Year Old.s, 1977-78) 



Variable 



SR(l-p) SR(3-p) 



Item Order. 

Standardized 
Residu^JL (l~p) 

o 

Standardized 
.Residual (3-p) 

Item Difficulty (p) 

Item Discrimination (r) 

Format (F) 



.1'8 



,30 



.09 



-.62 




.25 

.07 
.04 
.49 



^l=iMultiplerChoice ; 2=0pen-I^rided . 



■eric 



182 



-16:;- 



V 



Table 3.6.36v 

* Association Between Standardized Residuals 

and NAEP Item Content Classifications 
(Booklets , No. L and 2,^ 260 Items, 9 and 13 Year Olds, 1977-78) 

Standardized Residuals . 
Number l~-p " ' 3-p 
Content of SR(<1.0) SR(>1.0) SR(<1.0) SR(>1.0) 

Category Items (n=48). (n=212) - — (n=197) (n= 63 ) 



Sc©ry Problems 52 21.2 78.8 82.7 17-3 

Geometry 48 22.9 77.1 75.0 25.0^ 

Definitions 42 16.7 83.3 78.6 21.4 

Calcurations '83 • 15.7 84.3 69.9 30.1 

Measurement 17 11.8 88.2 82.4 17.6 
Graphs and 

Figures ^18 22.2 77.8 72.2 27.8 

0 ^ 

" ' ' 2.08 ' :<^== 3.65 

d.f.= 5 . p='.838 d.f.-5 p=..602 



183 
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Table 3.6.37 

. • Assoclacion Between Standardized Residuals 

?! and Item Formats 

; (Booklets No. 1 and 2, 260 Items, 9 and 13 Year Olds, 1977-78) 



Format 



Standardized 1-p Results 3~p Results 

Residuals N % N % 



Multiple-Choice SR(<1.0) 2^ 9.2 115 44.2 

" ■' SR(>i.O) 116 44.^6' 25 9.6 



Open-Endeci s\(<1.0) 24 • 9.2 82 31.5 



SR(>1.0) 96 36.9 38 14.6 

X^=.186 X"^ 5.98 

d.f .= 1 p= .666 d.f .= 1 p= .015 



184 
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Table 3.6.38 ; 



Association Between Standardized Residuals 
^ and Item Diffixulties 

(Booklets No. 1 and 2, 260 Items, 9 and 13 Year Olds, 1977-78) 



Difficulty 


Standatdized 


1-p 


Results 


3-p 


Results 


Level ' . 


Residuals 


N 


% 




. • % 


Hard (p<.5) 


SR(<1.0) 


14 < 


•5.4 


99 


38.1 




SR(>1.0) 


110 


42.3 


25 


9.6 


Easy (p>.5) 


0 

SR(<1.0) 


34 ■ 


13.1 


98 


37.7 




SR(>1.0) 


102 


39.2 


38 


14.6 
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Table 3.6.39 

Association Between Item Formats 
and Item Difficulties 
(Booklets No. 1 and 2, 260 Items, 9 and 13 Year Olds, 1977-78) 







Difficulty 
Level 


format 




% 

•a 






Hard (p<.5) 


Multiple-Choice 


70 


26.9 




















Open-Ended 


54 


20.8 ■ ■ - . 


• 




Easy (p>.5) 


Multiple-Choice 


70 


26.9 








Open-Ended* 


66 


25. A 






a 






.4'63 










^ d . f 


1 p= .496 
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Table 3.6.40 



Descriptive Statistical Analysis of 
Staadardized Residuals 
(Booklets No. 1 and 2 , 260 -Items, 9 and 13 Year Olds, 1977-78) 



^culty 
Level 



Fo rma t 



Number 

of 
'items 



1-p Re^sul ts 



SD 



3-~p Re,sult"k 



SD 



Hard (-X.S) Multiple-Choice 70' 

Open-Ended 54 



2.73 
1.64 



1.55 
.81 



.82 
.86 



.-23- 
.28 



Easy (p>.5) 



Multiple -Choice 70 
Open-Ended 66 



1.79 / 1.10 
1.'67 -.7? 



,90 
,97 



,64 
,38 



4' 
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Table 3.6.A1 



'Model 



Relationship Becween Icem Discrimination Indices 
and Standardized Residuals 
(Booklets No. 1 and -2, 260 Items, 9 and 13 Ye°ar Olds, 1977-78) 



Standardized 
Residuals 



■.01 'to .30 



1-p 0.00 to 1.00 

1 .01 to 2.00 
over 2.00 



( 29 )^ 
0.0 
0.0 
100.0 

143.7 
liLa= .691 



Discrimipation Indices 



.31 to .50„, 



.51 to .70 



( 55 ) 
10.9 

32.7 " 

56.4 

d. f-.= 



( 125 ) 
33.6 

62.4 
4.0 

P= 0 



.71 to 1.00 



( 51 ) 
0.0 
29.4 
70.6 



•J-p O.Op ta 1.00 

1.01 to 2.00 
over 2.00 . 



, 75.9 
■ 20.7 
3.4 

,'.■-= "5.28 
Eta= .092 



:>Iumber of test items appear- in brackets; 



80.0 
18 . 2. 
1.8 

d.f .=^6 



76.8 
23.2 
0.0 

p= .508 



68.6 
29.4 

2.0 



- \ 



TRIG 



1 8-8 
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Table 3.6^^6 
^ ^ 

• Relationship between standardized residuals and coAtent categories 

The pattern of standardized residuals is the same across content 
categories. Misfit statistics .for • both the one- and three-parameter 
models clearly are unrelated to the content of the test items. Of 
course, the standardized . residuals are substantially smaller for the 
three-parameter model. ^ 



Table- 3.6.37 

• Associaffon* between standardized residuals and item fo\rmats. It 
seemed useful to )inow whe-ther the pattern of misfit statistics 
for multiple-choice and open-ended test items was the^ same with 
the one- and three-parameter models. 



1. The pattern, of misfit statistics with the one-parameter modelN..s about 
the same with the two item formats. Residuals were somewhat larger 
with multiple-choice items .. 

2. The pattern of misfit statistics with the three-parameter model*was 

' also about the same for the two . item, formats . Somewhat surprisingly 
the results were a little poorer with the open-ended items. One 
conjecture is that, the c parameters were over estimated,^ 

Table 3.6.38 ' .. 



• Associations between standardized residuals and item difficulty. 

The one-parameter standardized residua^la were substantially higher 
for diffipult items tha^n for easy items. 

The three-parameter standardized residuals were unrelated to item 
difficulty, : ■ ^ o 



Table 3.6.39 

# Association J^etween item formats. and item difficulty, . 

There were approximately the ^s^me number of hard and ehsy test items,- 
and the distributiop* of items ifi .each format for hard and -easy items" . 
was about the same. .There were a few more easy o'pen-ende'd test items 
than hard open-ended te^st items, \ - 



^The problem was likely 'due to our failure' to designate some^test i^ms 
as "open-ended" when running LOGIST. ' . 



TabL. 3.6.40 



• Analysis of standardized residues for items organized by 
difficulty and format, " ' 

1, Hard multiple-chqice items had substantially larger residuals when fi 
by-, the one-parame^r mr^el than easy items in either format," cirr haTd 
.:|^tems in bpen-endea\^ormat4. This result^ suggests' that the problem' is 
due to a failure to a^'count for guessing behavior (note,, the fit was 
better for^hard open-ended items whe^e" guessing behavior is not / 
operative). The differences betweeni the ^average one-parameter ^n^ • 
three-parameter model standardized residuals, except for the_ hard 
multiple-choice test items, ^are probably due. to the difference in the 
Tway item disc'rimin^lting power is 'handled, .With the hard multiple- 
choice test items, qhe difference is due to a failure to account for 
both item discriminating power and examinee guessing behavioi?^in the 

^one-parameter model, . ^ *. . 

2. There were no relationships among item difficulty level, item format, 
and standardized residuals obtained from fitting the three-parameter i 

model, ' <7 \^ • '- ^ 



Table 3, 6, .41 - ■ ' 

• Relationship between item ^discrimination indices and standardized 
residualis , . > , • 

1» The one-parameter piodel resi^duals are non-linearly related to cl^ssl'c 
item discrimination indices (Eta=,691), * ' ^ , ^ y/ 

' * . / 

2. The^ three-parameter .model residuals are not related in any fashion 
• to^lassical item discrimination indices (Kta=r , 061) ' ' 
' ^ ' • " ■ I \ ^ " - f 

. - . ^ ■ \/'^" ■ ■ ^ 

In suiranary, the results of our hypothesis testing -showed clearly that 

r . \ ■ ■ ' ; , • 

the test items^in the content categories we -worked with were not in any 
way being fit better or worse hf the item repouse models, and failure to 
consider examinee guessing behavior and variation in item dd,qcrimina.ting, 
power resulted in" the one-parameter model _proViding substantially poorer 
fits i*tf^Ehe various' test data sets than tbe three-^parameter >cdel , ■ 
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4.0 Conclusions 

4.1^ Implicatlpns of Findings for NAEP 

The potential o\ item response thepry has been- widely documented 
but that potential is^certa-inly not guaranteed when applied to parti- 
cular tests, with particular samples o.f. examinees ^ or when used in 
particular applications. Item response theory is not a magic wand 
to wave over a data set to fix all of the inaccuracies and inadequacies 

in a test and/or the testing procedures . But > when a 'b^nk of content valid 

• i ! ' 

and technically sound test items is available, and goodness of fit studies 

reveal a highj match between the. chosen item response mod^l .and the test dats 
item responses models. may be useful to NAEP in test development, detec- 
tion of biased items ,^s^^e reporting, equating test forms and levels. 



item banking;, arttKother applications as well, ^The goals of this 
study were i^i a general way aimed at all possible item response model 
applications : to NAEP data, but specifically aimed at the possible 
..uses of .item Response models irl mathematics item banking-, one' of the 
less-fer important concerns of ECS on the NAEP project at the p'resent 

time. Still i there is great inte-xest at the national,, state, district, 

i ... 

I ^ . - • . 

and school-l^vel* in itepi banking and NAEP • e>tercises . In addition to • 

the' overall cuality of NAEP exercises, NAEP^ exercises' are "normed" 

^ • ■ ' • J. 

an3 Ijo interest in them and their statistics is high. * " • 

'^he implications of the present study for NAEP are the following: 



were 



/ 



1. A laige>number of goodness of fit Investigations were 
desciibed in Chapter 2 and several new investigations 
conducted and described in Chapter 3. Many of 



these , investigations can now be^jied .on Other NAEP 



data sets to' determine the generalizability of the^ 
conclusions drawn in this 'study concerning model data 
fit.^ » 

o 

The findings "o~f^ this investigation clearly support the 
desirability of conducting a wide range of analyses on 
a data- set, and on several data sets. We^ a narrow 
set of analyses to be conducted on (possibl^^) a single 
model_aijjd_^ia4:-^^&^^ th|e interpretation of resavjjts would 
have been more confusing and difficult. The approaches 
lescribed in Figure 2[^3.1 should provide^some direction 
t0;kNAEP staff and other researchers with an interest ; 
,<in applications. * ^ 

i Xt seemls^ that the three-parameter model performed 

I substanti^ly better than the one-parameter model. The ' 
results wer^not especially surprising, given informa- ; 
1 tion about thkways ±h which the NAEP exercises are 

constructed (i^^. , relatively little use is made of item 
statistical infoWiation id test ^ development) . .While the 
utility of the thr^e- over the one-parameter^ mo^el was 
not t<^o surprising^ the actual fits .of the threeyparafneter 
■ model to the^data sets were. . The study of Standardized 
residuals at the iteA level and abi^^ity level /revealed a 
very good fit of the three-paramet^t model. | • • . 

. Npt all of the analyses revealed high three-parameter 
mojdel-test data fit. .The* studies of "bias" we r^ the 
Wst confusing. Regardless of whether the-^hree- 
parameter model or the one-parameter model was fitted 
to'the-rdata; a number of potentially "biased" items were 
idervtified. Several possible -explanations exist: p 
Several test Items are biased- against one group or' 
ant>ther (e.^., race, or high and low perfonners) or 
th'ere'^are- problems in item parameter estimation (e.g., 
c parameters cannot be properly estimated in high 
performing groups or in any groups — black or white 
or hi%panic — if group size is of the size used in 
this investigation). , 

Perhaps • the .most important .finding is that it is highly 
'Unlikely that the 'One-parameter model will be useful 
with NAEP mathematics exercises. This is in spit^ of the 
fac t that m^hy other ^ organizations-* ^are very pleased with 
their work with. the one-parameter model.. With NAEP ' 
mathematics booklets it appears there is too much va'ri- 
ation among mathematics items in their /discriminatfn.g ' r 
power and too much guessing on the har^ multiple-choice 
test items f-or''"the""bne-parameter modeyL to provide an 
adequate fit to the test data. 



It is our opinion that the results from the first part of the study 
will be of interes"t and value to measurement specialists who are cun- 
sidering the usefulness of item response . models in their work. Essen- 
tially, we are recommending that measurement specialists design and 
carry out a comprehensive set of analysed to provide themselves with 
sufficient information to make informed judgments about the usefulness 
of item response models in their particular applications. The amount 
of effort extended in collecting information will be, of course, 
directly related to the importance of the intended applications. 

The second part of the study provides information that can impact 
on the future use of item response models in NAEP . There is considerable 
evidence in Chapter 3 suggesting that the three-parameter logistic model 
provides a very good accounting of the actual mathematics test results. 
The one-parameter logistic model did not . It may be that NAEP will now 
want to consider utilizing the three-parameter model in some small scale 
item bias, item banking, and test development efforts to determine the 
utility and appropriateness of the three-parameter model. Such investi- 
gations seem highJ.y worthwhile at this time. Of course, it may be that 
with other content areas the one-paramt-ter model may suffice, and for 
problems of score reporting new models being ceveloped by Bock, Mislevy, 

0 

and Woodson may be substantially better than the three-parameter logistic 
model. 
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Appendix B " 

Item Response Model Residual Analysis Program 
(Program Listing and Samole Output) 
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THIS PROGRAM WAS nESIGNED AT THE UNlVERSlTy CF M ASS /CHUSE T T S 
ON THE C3C GY8EK 175 IN THE FORTRAN VERSION 5 LANGUAGE 



PROGRAM TE ST (INPUT, OUTPUT, T A PE50, Tap E60, TAP E8, TAPt^ 
D.IMENSIOVJ' EX{l9r75) , A (75 ) ,B t/S) ,e<75) . ^ 

DIMENSION SE (19, .75) , OE ( 1 9, 75 ) » S V ( 1 9 , 75 r , (?E S 1 0 ( 19 , 7 5 ) 
D IMENSTON IR ESU L T ( 1 9 , 7 6 » , I A NS ( 75 ) , PReSULT ( 1 9 , 75) 
DIMENSION ITITLE (^3 ) , ABI L(19) 
REAL LEVEL 

CHARAGTER»13 DATE, TODAY .• 



OUTPUT 15 PRINTEC ON TA^E 8 
DA TA IS -READ - I N "ON TAPE 5, ••TAPE5u , 
Ta^£51 contains ITEM PARAMETERS 
TAFE6(J CONTAINS ABILITY ESTIMATES 
TAPE5 CONTAINS PRGGRAf OPTIONS 



T-APE60 



AND RESPONSE VEcTCrS FOR ALL EXAMINES 



'DIRECTIONS F0=> SETTING UP OATA DECK ON TaPE5 



CA RD 
CARD 
CARD 
CA RD 
CA RD 
CA RD 
CA RO 

CARD 
CARD 



1-READ IN USER DE-FINED TITLE (ITITLE) 
^-NUMBER CF ITEMS (ITEMS) 

3- NUM8ER oF ExA^lNEEg (NSURJ) 

4- MAXIMUM ABILITY VALUE (R^AX) 

5- MINIMJM ABILITY VALUE (RMIN) 
f.-Sl7E 0"^ AN ABILITY tATEGCRY (SIZINT) 
' " - TH i P VALUES (IPP) POINTED 

N FOR NO 

THE RESIDUALS (IPR) PRINTED 
N FOR NO 

ThE STANDARDIZED 
N FOR NO 



)0 YOU WNT 

Y FOR YES OR 

8- DO YOJ WANT 

Y FOR YE S 0 F 

9- 00 YOJ WANT 

Y FOR YES OR 



RESIDUALS (IPS! PRINTED? 



READ (5,520) ITI TLE 
52 (J FORMAT (23 A i*) 

WRITE HEADING PAGE "^OR U MA SS 



TOOAY = OAT: 
WRITE (3, i; 
1 1 FORMAT (iHl ,/ 
♦/,.5uX,"0ATe: 

'PROG'?AM ~ ' 



ERIC 



( ) 

)TODAY - . • - 

,///////, 35X, ••RESIDUAL ANALYSES OF LOGISTIC 
,.A10,/ ,5. X, VERSI CN 3A ) •',////, A C X , 
»"PROG'<AM TY LINDA M URkA Y " , / , 5i X , "R 0 NAL D HAmBLETCK", ^ 

SIX , "ROBERT SIMCN", /////, 35X, 
»"D'^VELOP-n AT THE UNIVERSITY OF MA SS AC H U SE T T S" , / , ^iX , 
• ♦"SCHOOL OF EDUCA T ]0N", /, t.2x, " ~ 

♦"UNDER A GRANT. FRCM NaEP") 
WRIT£(fl,530) ITITL f 
130 foRimAT (/// /,35 X ,20 A^ ,/// ) ^ 
R£A[}(i5tl) ITEMS^ 
READ(5,1) NSUBJ 

1 F0RMAt.(I7) 
READ (5, ?) RMA'X 

2 FqRHAT (PS, 2) 
RfAD(5 ,2) RMIN 
READ(5,2> S IZINT 

' ' rEA0(5,3) I P? 
READ (5,3) IP^ 
FORMAT (All 
READ(5,3) I PS 



TEST DATA' 



20,3 



C LJMIT. IS THE NUM3ER CF ABILITY CATEGORIES 



C 

c 



LI>1IT= ( A3S (RMIN) ♦RM AX) VS IZ.INT 



WRITE (8,21 ) IT EHS.N SUB JtLIMiT 
■ 21 FCRMAT(/////»35X ,13 'ITEMS / ",1,5 , •••EXAMI^EES / 
'♦12," ABILITY GROUPINGS "> - 

C MOCEL i IS THr Of>^E P'AFAHETER LOGISTIC KdCEL ' . • ' . " 

c • . - ■ • ■ . 

C 

M00EL=1 ' . ' , 

' "'8 ITEKONE IS. THE L A> T CCLUHN IN THE O3SERVE0 p VAl'dE MATRIX 

. c FCR Counting the number people in an ability category • 

^ I lEMO-NcIl T £MS*-1 , • " 

c ■ . . " ■ , 

C READ IN FOR MOfEL 1 CN TAPE6C A,B,C PARAMETERS FOR ALL ITEMS 

. 00 k 1 = 1. ITEMS ^ . ' • - • 

REA0{6G,2,U) A ( I) ,B (t^fC (I» 
20 0 FORMAT (oy,3(F6.3)» , , 

h CONTINUE. ' 
. , ■ >G ( TO i:i • 

5 C CI;JTINUE • • . • 

• REWIND &: . e . ■ ' 

;•• c' . 

C REAC- IN FOR MqCEL-'3..CN TAPE60 THE A,B,C PARAMETERS FOR ALL ITEMS 

■* C ' • ■ \ ' ;/ ■ 

• C MODEL J- IS THE' THREE PARAMETER LOGISTIC MODEC 

C • 1 • \ .. • ■ . 

M00EL=3 

0.Q 6 1 = 1 r ITEMS " ■ 

^. REAOteC, 7') A ( I), 8 (1) ^C (I ) 

7 F0Ry-AT-(,2i*X ,3 (F6. 3 ) ) 

6!- GONT;INi}E ^ . • 

V • '^0.^0 ,10' ^ ' . 

8 CtCMTINUE' . 

c • ^ • ■ 

• C This is TPir termination OF THE RROGRAK •. ' . ' " ■ " 

C rjHEjC COE , OIlO'-! THiS FOInT IS USED TO CALCULATE^ THE P VALUES AND RESICUALS 
C / . # 

. to CONT-imj£ ' • ■ . 

• C 

.C Ar is THE-MlDocif^ OF THE LOWEST ABILITY CATEGORY 
,^ V8'=rS[/iM-5 I*ziNT/ 2.'J ) » 

C THIS loop" ^AboOLATES THE MIDPOINT OF THE ABILITY C AT EGOR IE S A BI L ( J ) , 
C AND THg,.E xF^E:TED P VAlUES MATRIX, EX(J,^I) 

^ 00 30. • 1=1, ITEMS ■ . • 

. 00 IL'i J=1,L WIT . . . ... 

A-niL( J),= Ae+ (J-1)*SIZINT . . ... 

. QD= (1 .,7*A ( I) ^ ( AP KU )-B ( I» > ) 
0 = 2.7l82'8i 8»»( 0D> ' . 

* • EX ( J, I)=C { I),«-( 1 .0-C ( I) » • (0/ (1 .0 ♦O) > . ■ 

IF(EX (j,I) .lT. /G1>EX(J,I ) = . 01 
O .IF (FX<tl,I ) .GT. .,gg)£X(J,I )= . 99 904 

ERI0«" CCM.INUE . 



C S'-T-Pcftl TO 7 g'rtf THE GQUN^^R FCR 0£^p RM I NI N HE Nu'^SER^OF tXAHI.'^ES 
C i.^ I r^^(Jt^^-'i^3fj^yjY EsflMATcS BEYONC TH£ CHOSEN RA'Nfjt 




c ■ With 43 

^ KCOUNT=. 

C Z'fiC GUT ARRAV IRESULT WHICH CChTAlNS T-H E NUf^BER OF EXAmInEES IN AN fBlLIlY CATEGORY 
C ~ AUC THE UJ'^BER /Of EXAmInEES WHO GET THE ITEMS CORRECT 
C 

00 i»flvI = l»LII 
0 0 uS': J=l . IT! 
IRESULT (I , J) 
U8 0 CONTINUE v ' 

C Pc-AO IN FOP EITHER MCCEL ON TaP£5C THE ABiliTY ESTIMATE (LEVEL) AND THE RESPONSE " ■ 

C VECTqF? for each EXAMINEE (IANS(K>) 

Co - - ,. •• 

00 1l- I=1.NSUBJ 

1 F ( MODEi • E 6- 3) Gn 10 15 

READ( ^0 ''♦•■3 0> LEVEL, ( I A NS { K ) ,K = 1 , I T E MS ) 

GO TO 19 

I 5 CONTINUE ^ 
REAQ(5j,to )LEVEL, (lANS (K ),K=1,IT£MS) 

16 F CRmAT (/, iOX.Fi b. 2, 2X, 73 II ) \ 
19 CONTINUE \ 
U90 F CRMAT (/, F ifl. 3,1 »JX, 2X, 73 11 ) \ 

C KCCLNT COUNTS tP THE NUMBER OF EXAMINEES THAT FALL BEYOND T-H-E-^-vMA XI KU 
C 0 1 MINIMJM ABILITY VALUES - #•'■> 

C ^ 

IF( (LE'V-L- GT. RM AX) .OR. (LEVEL. LT.RMIN) )Kc CUNT =KCCL'NTtl 
IF( (LtVtL. GT.RmAX ). OR. (LEVEL. LT.RHIN) )G0 TO 11 C 

C lABIL IS THE AGILITY ESTIMATES TRANSFORMED INTO AN ABIlTY CATEGORY ^ 

r ' I 

IARIL=(( ( L EVEL+A BS(RMIN) )/SI ZINT ) +1) » 
IRESULT (lA E IL, lT£fONE) = IRt SULT ( I A R I L , I T E ^GN E ) +1 
00 2C J = l . IT EMS 

IRESULT(lAgiL»J>=I'^ESULT (lABlLfJ) fIANS( J) 

2C CONTINUE 
IJO CONTINUE 

C 

C,PRESULT IS T'-^r. MATRIX OF OBSERyED P VALUES - 

^ GO 22 1 1=1 .ITEMS - ,^ • 

fl' PR^IuLTUri ) = 7F\oAr (IRESULT (J, I ) )/FLOaT { IRESULT (J, iTEmGnE) ) ) % 

I 22 0 CONTINUE 

\ • REwif^n 5j ■ 

C DO VOL WAMT rO PRINT OUT THE P VALUE T A T LES ????-????? ??? 

C IF ■ SO IPP IS - TO Y 

C ' 

IFdPP.NE. "Y-JGO '3 w71 

C 

C PRINT THt EX='eCTED P VALUES MATRIX 

.-(RITE ( 8,?5C> MODE L 

WRIT^ (8,110) 
llJ t^CRM. " (^j&X ,'ABIL ITY LEVEL') 

WRITE ( I 2 G» (I , I =1, L IM IT ) 
1' j FCM.' (3X , "CATEGORY "MX, 19 (3X ,1 3 »/ ) _ „ 

Hi FORM, :.(iMi ,////, :,0X, 'EXPECT ED P V A LU E5 - ' , 1 1 , ♦ pAKAMtTER MOq^L',//// 

2FRir '"'vRITEO.'S^ ) (AB IL(J » .J^l .LIMIT ) 206 

i^ta 55 FORMAT (/, 3 X,"Min-POINT", 2X. 19UX,P5. 2 »/ ) 



• WRfTE'l'i,liQ ) (IRESULir I, ITEMCNE) , 1 = 1 jCIMIT) ^ ^ 

WRITP (3,1-j Q) 
; 150 FCRM5T I, • ITEt" ) 

' L = i& , ■ • 

DO - i = ltITtMS 

WRITE ( 9,!»5UU , (E X(J ♦ I) ♦J=i »L IHIT ) 
'♦SO FCRMflT(fy , 12, 6X, ic( rf,, 3) > 

K=i03(I,LI 

IF(K.ME.O)Go To ^51 

WRITE Ci, 752) 
'♦SI CONTINUE 
k(i t Co'^JTIMUE 

c 
c 

C PRINT THE OBSERVED P VALUES MATRIX 

^ WRITE f 8, '■♦70) 

'♦70 FOR-^ATC//// ) 
■WRIT£(8,12) 

WRITE(9»510)MOOEL 
lu 2 WRITE (8,110) 
115 WRITE(3, 120) M , 1 = 1, LIMIT ) 

WRlTc(8,55) (A B IL <J ) ♦ J=l tL IM IT ) 

WR ITE ( 8, 1 30) (IRE SULK LITE MONE) , 1=1, LIMIT) 
130 F0RMaT(3< , 'NC. CF'./»2X»' E X A MI NE E S * , 1 X , 1 9 ( 3 X , I 3 ) // ) 

WRITt(8,15U) 

fa FORMAT (////, 'THE NO. OF EXCl UOEO CASES=',I9) 
12 FORMAT (IHL ) 

5lC FORMAT(////,i*:' X, 'OBSERVEO P VALUE S U , ' PARAfElER MODEL',////) 
00 27. 1 = 1 ,IT£.*1S 

WRITE(8,26L) I, (PRESULT (J ,1 ) ,J=1,LIMIT) , 
260 F0RMAT(6X, 12,6 X, 19( F6. 3) ) 

K=MOO(I,L> 

IF(K .NE. 0 ) GO TO 261 

WRITE (8,252) 
262 FORMAT(/) 
26 1 CONTINUE 
2^0 CONTINUE 

WRITE (8, 1 7 C) KCOUNT 

C 

'♦71 CONTlrJUr 

C GO no RESIDDaL^S and STO RESIDUALS FoR THIS CASE 
G 

CALL ^(ITr rs,LIMIT, EX, IRESU IT, PRESULT, I T £ MO SE, M CCEl , 
♦ IFR, IPS, Al IL, NSUGJ, KCoUN T) 

C 

C IF ONE PARAM-TER MOCEL GO BACK A NO DO THREE PARAMETER MODEL 

IF (McoEL.EQ.i) Goto 5 

C IF THREE PARAMETER MCOiL GO TO END OF PROGRA ^' 
IF (MODEL . Eq.3) GOTO 8 <a 
END 

Q 

C THIS SUBROUTINE CALCUlaTES RESIDUALS, STD RESIDUALS AND VARIOUS STATISTICS 



C 



SUBROUTIM- R { ITEMS, LIMIT ,£ X ,IR , P R , I T EMO N E ,M 0 OE L , 
♦ I fS , If^R,A3 IL ,NSUaJ, KCOUN T) 
D I.-1ENSI0M Ss- (19, 7E) , OE (1 9, 7^ ) , S V 1 1 9 , 75 ) , RE S ID ( 19 , 75 ) 



n I MENS I ON It? (19, 75) , OR (1 9, 75 ), Ex ( 19 ,75) , APIL (l9 ) 

iREL-= : 

IRE1 = 
IR£2=: 
O . IRE3=. 

ERIC 207 



• • • t 

c sv IS The va^ianCe of" the expecxeo p values 

C S£ IS THE STANDARO ERROR OF ThE Ex:p£CTED P VALUES 
C OE IS THE RESi OUAL=0 B SERVED- EXP £CtEO 
C RESIO IS THE RESIDUAL STANDARDIZED 
C 

00 90: I=ltITEMS 

00 901 J=l ,LI IIT 

SV( J, I) = (EX( J. I )♦ (1 .0-EV ( J, I ) ) ) / FLOAT ( I fi( J, ITL KCNEM 
SE (Ji I)=SQRT(SV( J,I ) ) 

oE(Jii) = (PR(j,i>-^(E>;u,;:))) 

RES 1 0 f J , I) =0 c ( J ,. I) / S E ( J , I ) 

IF((AnS(R£SIO(Jt I») .r,£.u.juC>.AND.(ABS(RESIC(J,I)>.LT.l.GCO) ♦ 

*IPE:Ot:IREO ♦■I 

IF ((flRS(RESlO(J,in.GE.l.CaJ).AND.(ABS(fiESIC(JtI)).LT.-^.G00») 
*IRE1=IRE1+1 

IF ((ABS(R£SID<J,])).G£.2.0G0)%AND.*;ABS(R£SlC(JtI)).LT.3.uU0)) 
* I«^E2= IRE2* 1 

1 F(A8S(RES lOtJt I) ) . Ge. 3.;ulO ) IR£3 = IRE3+1 
90D CONTINUE 

P£Ru = (FLOa TdREO )/FLOAT( IT£MS*LIMIT) )»1CG.00 



PER1=(FL0A T(IREi»/FLOAT( ITEhS'LIKIT) )*1Q0.Q0 

- ."2>/FLL 

PcR3=fFLOaT(IRE3)/FCOAT{ITEMS*LIMIT»)'lCt..i;o 



PER2:^(FLOAT(IRE2)/FLOATfITEhS*LIMlT)>*lL.tJ.U0 



IF( IPR.NE. "Y'MGOTO 915 

C 

C PRINT »ESIDUAL MATRIX 
C 

C PRINT RESnUAL HEADING 
C 

WfilTE (8,90 1» MODEL 
93 1 FORMATdHl ifOX, 'RESIDUALS- '*I1,' PARAMETER MODEL',//, 
•<*5X • (OBSERVED-EXPECTED) S///) 
WRITE (8, 110) 

WRITE (8, 91 1» ( I, 1 = 1, LIMIT ) 

WRITE(6,17 35»(ABIL(J),J=1,LIHIT) 
1735 FCRMAT(/,3X, 'MID-POINT', 2X, 19(F 7.3)/ ) 

WRITE (8, 1 7 36 ) ( IR (I, ItEMONt ) , 1=1 , LIHI T) 
173 6 FORMAT ( 3X, 'NO. 0F',/,2X,' E X A M NEES ' , 2X , 1 9 ( 2 X , 1 3 , 2 X ) // ) 

WRrT£(8,15a» 

C 

C PRINT P;:-SnUAL TABLE OF VALUES 

c - 

CO 9i: 1=1 , ITEMS 

W3rTF(5,26 0)I,(CE(J,I»,J = i,LlMlT) . 

K^MQO(I,L» 

IF(K.fJE.O ) GO TO 9L'9 

WRITE ( 8.91 't) 
9J 9 C CNT INUe 
91 U CONTINUE 

C 

C EVALUATE STATISTICS ON THE RESIDUALS 
C 

CALL STATS(0E,LIMT, ITEMS, ABIL, IK,IT£MONE,NSUBJ,KCOUnT) 
WR iTc^ ( ^» '♦7 0) 
915 IF( IPS.NE. "r") GOTO 9^^' 

C 

C PRINT STANDARCI7EC RESIDUAL MATRIX ' 
C 

C PRINT HEADING FOk ST C RESIOUALS 
C 

W^ITE(5,9l2) JOEL 
O'iZ FORMATdHL ,////, /4OX , 'STAfjnc f;CIZ EO RESIDUALS" ',11,' 
FRir" "PARAMETER MODEL',////) 

££!^ WoxTE(5,liO) 2*^3 



WFrTE(8,9ll) ( I, 1=1, irriT )■ 

WRITE (8, 17 35) ( A 8 I L ( J ) , J =1 , L IM I T ) 
WRITE (8t 17 36) ( IR( I , ITEM 0M£ ) ,1 = 1 tL IM IT) 
WRITe(8,L50) 
911 FORMAT (nX, 19(2X » 33, 2x)/ ) 
00 qZC 1=1, ITEMS 

WRITE (8, 2 6 0) I, (RcSTO(J,I ), J=l,LIMlT) 
K = MOD (I ,_ ) 

IF(K. Nt. u ) Go To 913 ' 

WRITE (9, 91 i4 ) 
913 CONTINUE 
gi't FCRMAT(/) 
920 CONTINUE 

WRlTE(8,9Eli 

.921 FORMAT (////, lOX'ANALYSI S Cr STANDARDIZED RE S lOU ALS ' . / t 
♦18X, • (A3S0LUTE V A LU £ S 1 * . / v X , 

•'INTERVAL Number ^^ERCt-NT cumulative »,/, '4X,30X, 'PE rcent ' ,/> 
TOTPeR=P£R0 

WRITE (8,922) n EC ,PERC , TO TP ER 
92 2 FORMAT(6X,*0 TO i ' , 3 X , 3 X ,^^6 . 2 , 6 X, F6 . 2 , / > 

T0TP£R1 = 3£ Rj f PERI 

WRITE(8,923)lREl ,PER1, T0TP£R1 
92 3 F0RMAT(6X, '1 TO 2 • , 3 X , lu , 3 X , F 6 . 2 , 6 X , F6. 2 , / ) 

TOTPER2=3£R0 tPER i*PER2 

WRITE (8,9 2'i) IRE2 ,FER2, T0TPER2 
92'* FCRMAT(6y,'2 TO 3 ' , 3 ^ » I ♦ 3 X , Fe . 2 , 6 X , F6 . 2 , / > 

T0TPER3=3E Ru tP ER 1 ♦P E R2 4-P ER 3 

WRITE (8 ,9 2 5) lRE3, PER 3, TO TP ER 3 
92 5 FGRMAT(6X, 'BrYONC I ' , li* ,3 X ,F6 . 2 , 6X , F6. 2 ,/) 

C 

C EVALUATE STATISTICS FOR ST D RESIDUALS 



C 



C 
C 
C 
C 
r. 



CALL STATS (RESI C,LIMIT,rTEMS ,ABIL, IR,lTEf'CNE,NSUej,KCOUNT) 

9t»3 CONTINUE 

11 ( FORMAT (56X AB IL ITV LEVEL') 
15 0 FORMAT {/, 5X, «ITE H' ) 
1*70 fORmAK////) 

260 f^GRMAT ( i X , I 2 , 5X , 19 ( P 7 . ,3 ) 1 

END 



SUBROUTINE S TA T S ( RE S I H , L Im I T , I T t MS , A 8 1 L , IR , I TE K CK'E , nSU B J , KCO UNT ) 
3I.MENSI0N AVELIf(19),APSLIN(19),RKSLIM(19),R£SJC(l9,75) 



D I MENS I OS AVEITM ( 75 ) , AR3 IT M( 75 > ♦ R I'-iS I TM ( 7 5 ) ,A BIL (19) 
D IMENSION IR(19 ,75) , WA VI Tm (7 5 ) ,WA3lTN ( 75 ) 
WRITE ( «, i«7 1) 

C 

C ZERO 1HE VALJ-S FOR STATI-3TICS 



C 



00 1" IG I =1 , 75 
AVELIM( I ) =0 
AOsLIMd ) =C 
RHSLTMd ) =D 
AVEITM ( I ) =C 
A B S I T M 1 1 ) 



— L 



ERIC 



WA8irM( I ) 
O LGiO CONTINUE 
AVEAVL=: 
AVEAaL=D 



Rhsitm(i)=: 

wavitm( i) =c 20 j 



AVcAVI = j 

AVEAni=3 
AV£RML=0 
■ - AVERMI = t:< 
AVEWA-V=0 

avewab=o 

c calculate for average arility l e vel " a v e ra ge absolute ability level 

"-. C 

00 1 . 5u I =1 ,LIM IT 

DO l-.':0 , iTEf^S 

A Vr LIM( I) = AVELIM ( I ) +RtSI D f I, J) 
AeSLIH(I> = ABSLIM(I)+AES(RESIC(I.Jn 
ICOO CONTIMUF 

AVELIM(-I) = AVE Lie ; I) / FLOA T( I TEMS» 
A6SLIM(I)=ABsLIM ( I) / FLOA T( ITEMS) 
AVEaVL = AVEAVl ♦A VEi I'i( I) 
AVEABL=AV- ABL«-ABSLIH(I) 

1C50 CONTINUE * • 

A VFAVL = AV£ AVL/ F ICAT (LIMI T) 
AV£ABL=A\/EABl/FlCAT (LIMIT) 

C 

C CALACULATE ROOT MEAN SQUAFE FOR ABILITY LEVELS 
C 

00. 120 0 1 = 1 ,LIMI T 
DC 11-iC J=1,ITEMS 

RhSLIM( I) = KMSL IM (I ) ♦■((RE SID (I , J » - A VE L IM ( I ) ) • *2 ) 
lluO CONTINUE 

Pm'=;LIM( I) =SnRT (R^SL IM( I) /FLOAK ITEMS ) ) \ 

A\/ERML'=A'yERML«-RMSL IM (I » \ 
123 J CONTINUE \ 
AVERML=AVERML/FLCAT (LIMIT) «. \ 

C CALACULAtE FOR AVERAGE ITEM STATISTIC » AVERAGE ABSOLUTE ITEM STATISTICS 

— c 

00 C 1=1, ITEMS 

00 130 0 J=l,lIhIT 
A ITM (I ) = AVEITM ( I ) + RESI 0( J, I) 

ABSjTMd) = AOSIT M( I) ♦APS( SID ( J, I » ) \ 
133 0 CONTINUE \ 
AVEITM(I) = AVElTM(T)/FLOAT(LIMIT» ^ 
A ESITM(I) = ABSIT^( I) /FLOA T(L IMIT) 
A VEA VI=AVE AVI ♦■AV £ IT M ( I) 
A VEABI = AVE ABI»-ABSir.M(I ) 

1£,3 0 C CNTINUE . 

A VEAVI=A7E AVI/FLCAT (ITEMS) 
A"VEABI = AV£ ABi/Fl CAT ( IT E.'^ S) 

c 

C CALCULATE ROOT MEAN SQUARfc '=^0'? ITEM STATISTICS 

C . 

00 16. u 1=1, ITEMS 
CO 15vO J=l, LIMIT 

RMSITM(I) = RMSITM (I) +('(RESI"D ( J , I ) - A VE ITM ( I) ) * ♦ 2 ) 

150 J C (NTINUE . , 

RfSITM ( I) = SORT (R^<SITM( I) /FLOAT (LIMIT.) ) 

AVERHI = AVERMI«-RmSIT M (I ) 
163 0 CONTIMUF 

Av ERf^I^ AVE RMI/FL CAT ( ITEMS) 

C 

C CALCULATE TH^ W £ IG H T t C A V ERA Gc" R£SIDU,".S AND 
C WEIGHTED AVERAGE ABSOLUTE Rr.SlDLALS 

O KTOTAL=NiUej-<CGUNT O \ A 

FRir" 00 1 65LV.I = 1, I tehs ^-a-^' 

EA>*v CO lei+j J= 1 ,Li fi T 



MAy ITM(I) = Vir K (I) f.(R£SIO ( J , 1 1 •FL CA T ( IR ( J> ITE «0 NE ) ») 
W^TITHd) = WA9IT DXABS (RE SIOC J » in *FLCaT ( IK( J,ITEMONE) )) 
IbkQ CCNTIS'iJE 

WAVITKI.)- WAVITM (I) /FLOAT(KTOTAL ) 
WA3ITM( I) = WAg IT f (I) / FLOAT ( KTOTAL ) 

avewav=av/e: wav>waviTm(I) 

AVEWAn=:Ay£ WA3*WA P IT M (I » 

165 Q CCNTIMUE ^ ^ 

AVEWaV = A\/£WAV/FLGAT (ITEhS) 
AVEWA3=AV£ KAB /FL CA T (ITEMS) 

C 

C PRINT OUT FIT STATISTICS FQf? ITEMS 
C 

170 0 FORMAT (ixi'^UMMARY OF FIT STATISTICS FOR lTEf^S't//t 
»5X,'ITEM AVERAGc AVERAGE /iBSOLUTt ROOT MEAN',8X, 
♦'WElG^TEi AVERAGE WEIGHTfC ABSOLUTE ' t / , N 
*5X;' P.ESinUAL RESIDUAL SQUARE RESIDUAL 't6X, N 

" fESlOUA. ^AVERAGE RESIDUAL') 

00_^.7^5 I^. 1 ^ t^^^ J. ^^.j J , f'S^I Tt^ i t ) rKA V I T-K( t| rWA B'l TM t M 

1710 FORmW (6X, I2;2r,F7. 3t5X, F7 . 3 t 9X , F7 . 3 , l^* X ,p 7 . 3 , 1 1 X , F 7 . 3 ) 
K=H00(I,L» 

IK(K. N£. ; > GO TO I7j 5 
WRITE rs, 17 02) 
17)2 FORMAT (/) 
170 5 CONTINUE 

171 5 CCNTIMUE - - - - 

C 

C PRIM AVtPAG£S FOR ITEM FIT STATISTICS 

^ WRITE (3, 17 20 ) A VEA V I , fl V EA B I , A VERM I , A VE W A V, A VE W A 8 

1720 FORMAT (/, 2X, 'AVE RAGES' ,F-7. 3 ,5X,F7.3t 9X, F7. 3 » 
♦l'«X,F7.3,. iX,F7. 3) 

C 

6 PR INT ABILITY LEVEL FIT STATISTICS 
C 

WRIT£(8,^71) 

1721 FCRMAMU-Oy^.'SUMMAfY OF FIT STATISTICS FOR AGILITY LEVELS',// 
*5.X, ' ABILI TY LEVEL( MID-POINTS) /> 

WRITF( '^,'1,7 50 ) (I, 1=1 ,LIMIT) 

WRITE(8,1^22) {ABIL(J»,J^l,LlMir) 
17'2-FCP.'^AT (1 V, 'FIT ', IV ' S T A T IS T I C ' , 1 5 X » 1 5 ( F 7 , 2 , 1 X ) ) 
1753 FaRM^-T (25X ,15 (5X ,Ij ) ,/ ) 

WFirE(%l^(«) (A vELIM( I) ,1=1 tLIMiT) 
1760 FORMAT (IX, 'AVERAGE R ES ID UA L ' , 1 2X , 1 5 ( F7 . 3 , IX ) ) 

• WRI TF ( 3, l 770 ) ( ABSL IM( I) , 1= l.LIMIT) 
1770 FORMAT (//, iX, 'AVERAGE ABSOLlTE 5 ES ID UAL ' , 2 X , :.5 (F 7 . 3 , 1 X ) » 

^ -WRITE (8, 1 760 ) (Rf-SLIHTl ), 1=1 LlKIT) 
178J FORMAT (//, IX, 'ROC 1 MEaN SQUAr£ r ES ID UAL ' ^ ■ ^ ^ ^ t - v » 1 

H PRINT AVlRAGES for ABILITY lEVEl FIT STATISTICS 

WRITl- (8, 71 ) ^ . 

WRITE (8, 1 790 ) aVEAVL, AVEAOl tAVERf^L 

179 0 ^SR'^lAT Isx'! '^VF-RALL VtRAGES AVEpAGG RESIDUAL=^ ',F7.3,3X, 
»'AVFRAGE AESOLUTE RtSIOUAL= ' , F 7 . 3 , 3 X , ' R C CT 
*• RdSlOUAL= ',F7.2) 
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l.OOG 1.000 
l.OCO 1.000 
.886 1.000 
.977 1.000 



. 60 C 


. Cli* 


• 886 


.95 9 


. 9U1 


.977 


.96 3 


. «83 


.977 


. 87 3 


. 89? 


M 


. 780 


. 833 
. 833 


.678 


.955 


'Ml 


•225 
. ^8C 


.227 
.5<»5 


.82 0 


. «<I2 
. ^80 


.909 


.98 8 


l.CCO 



•H2 
. 735 
. C51 
. 9i»l 
. 735 
. C61 
. t?k 
. 9^0 - 



.909 
-1.-000 



, 000 
.00 0 



} 
I 
i 
J 
} 
} 
J 
i 



218 



ERIC 



RESIDUALS-- 1 PAKAMEieR HOOEL 
(OeSERVEO-EXPECTEO) 



ABILITY LEVEL 



MIO-POINT 

NO. or 

EX/iMrN£ES 

ITEM 

1 . 

2 

3 

5 
S 
7 
8 

9 
10 



I 


2 


3" ~ 


— u ■■ ■ 


5 


- -6- ^ 




8 


9 


10 


•a .75 0 


- 2. 25C 


-1.75 1 


-1. 250 


-• 750 


-. 250 


.2*50 


. 750 


1 . 2,,?U 


1 .75 0 


lu 


5^ 


91 


22<« 


325 


50 3 


^♦67 ■ 


33 9 


2 45 


102 


. 00 3 

ceo 

. C2 2 
- . 

-. Cl'6 
. 05 3 
. 116 
• <*5 7 

015 
COO 


- . 2li 5 

- . C57 

- .025 

- . C5i 
. 075 
. U65 
.37£ 

. :5 9 

- , 11 2 


-. 131 
-. 090 

. i5e 

. C9d 
. 26<* 
. 075 
-. 13« 


-.069 
..OOf, 
J06 

-.137 

-.073 
.079 

. .072 
.173 
.'025 

-.105 


-.011 

.001 

.a:i5 

-.1 Cl 

.oie 

.018 
.091 
.016 
-.0 cZ 


.039 
.011 
.055 
-.C75 

- .010 

- .Q3C 

::m. 

.062 
.009 


tj 

.022 
.00 9 
.012 

'.86^ 
-.03 5 . 
-.0<»9 
-.005 
-.018 

.0*4 5 


.0 29 
.0 05 
.0,06 

. 1, 50 

,.n72 

-.1)30 
-. 0<42 
- .103 
-.070 
.0 <*3 


-.0 Jl 
-.331 
.032 
.1^9 
. 0^3 

:\Ul 

-.076^ 
-. 0 62 
.0 27 


.016 
.010 

:m 

.033 

-. Q 7*4 

f013 




-.0 13 
.010 
.010 

.075 

:IU 

.028 
.013 



• 010 
.010 
.010 

• 03S 

• 021 
.087 

• 070 

• oie 

217 

> 0 i C 



11 
12 
13 

1*4 

15 
I 6 
17 
18 
1 9 
20 



. 55 7 
» 39 5 
. kl9 
, 39 9 
. 22 7 
• 278 
, 296 
.010 
I 33 5 

. la*4 



. :92 

. i6*4 

■•Ml 

• . U 9 
. I3u 
.122 

(10 

• .I 'j 9 

• . J95 



637 
08 9 
121 

029 
IJ 5 
03C 
12C 

, 01*4 

160 
15 J 



.013 
.9Q3 
,026 

.02; 

.010 
.019 
.3l3 
.316 

. l5*4 

.175 



.022 

:m 

.00^ 

. oca 

.010 
.002 
-.0 12 
-.118 

193 



.016 
.0 21 
.0 27 
.w 11 • 
.OC7 
.0 31 
.G 5<* 

-.111 
.C 62 

-.013 



.016 

.01^4 

.026 
.016 
.U2Q 
•.Cu2 
-.057 
.025 
.063 
.097 



.0 05 
. Oil 
. 006 

-.0 18 
-.0 ?5 
-.005 
.0 <»6 
.152 



. 0U6 
.03 2 
.001 
.30 8 
.032 
,312 
. 0*42 
.017 
.0<*9 
.087 



.010 
.0 10 
.010 
.010 
.011 

C58 
.05^ 

. 0<4l 

.080 



.0 iO 
.010 

.010 

-.088 



.010 

• 010 

•Ml 

• 010 

.010 
.0 13 

• 30<» 

• 02<< 
.027 



1^ 

23 

2*4 

25 
26 
27 
28 
29 
30 



, 056 
0*45 
121 

: 018 

, 025 
I 151 
C 61 
OS 1 



' . J77 

• ill 
. J99 
• . C60 
.377 
. J I 0 

, 11*4 



112 
, 09*4 
10 1 

13<4 

010 
. 22 5 
OC 1 
116 
01 0 

: CQ7 



-/l7fl 
-•;aC5 
. 161 
. 08l 
-.097 
. 355 

- .00*4 

. 1U5 
01 e 
.3*4 7 



-.162 
-.009 
. 379 
.003 
-.349 
.0*46 
. QC*4 

.fl9e 

.fl05 
-.097 



' .G 10 
•.C02 
.003 

• .C 1*4 

• 0 27 
-.050 
.102 

• .0 1*4 
> ,1*46 



.06*4 
-.013 
-.059 
-.025 

.035 
-.05 8 
..001 

.028 
-.037 

.014 



.1 39 
.002 

- .1 0*4 

-.0 i^l 
.0C2 
0 16 

. J 3*4 

•0 l5 



. 066 
.033 
.03<» 
.056 
.018 
.217 
,051 
.056 



.095 
.000 
-. 023 
.0<»2 
.026 
-. COO 
-. 026 
-.207 
.001 
.061 



.057 
.010 
.072 
. 097 
.023 
.023 
. 162 
-.352 
. Q6<» 
.Ogl 



. 0«5 
• 101 
. 0<»0 

-.1^9 



31 
32 
33 

3*4 

35 
36 
37 
38 
39 

*4»i 



06 8 
C8 1 
029 

: O7 9 
33 3 
133 
022 

, 18 0 
315 

, C23 





. -. 016 


-.066 

':3 5 


-.0 25 


. l y^ 

. 0^2 

. Wv 5 


. 073 


-.059 
.059 
.017 


. q79 

..2*4 7 


.62/* 
-.020 


. c7\i 


.195 


.u93 


.026 


. 13 3 

. 055 




.ififi 


.183 


.u53 


.037 


- .205 


-. 283 


r^^.ii c 


-.0 58 


.131 


. C68 


.121 
-.0/2 


.075 


- . n*4 


-. 1j6 


U*47 



-.003 
►.065 
.002 

> .001 
- .0 58 

.U 8^ 

► .C05 
.0 53 
.03^ 

-.021 



.025 

-.006 
-.065 

.0 10 
-.020 

.053 
-.0<»2 
T .051 



,a 19 
,0 f 3. 
. 002 
,0 13 
.0 39 

,1*45 

,015 

.0 52 
.3 95 
,3 61 



.008 
• 0 83 

r. 096 
.006 
.0 36 

-. 239 

0 

.0^*0 
-.088 
.029 



.036 
. 068 
-. Cl<» 

:m 

-.391 

^.03*4 
.028 
090 
. 0<»9 



-. CC2 
.029 

-.0 (6 
.010 
.033 

-.396 

-.0*43 
.016 

-.066 
. 03<4 



• 012 
-.303 
,062 
.010 
.019 
-.81<» 
.0(4 
.010 
-.318 
.020 



»4l 



U13 



C15 



.00 6 -. 00 2 -. 0 *4*4 -.0 29 -.OjC -.016 



.065 



.145 



.106 



• 138 




. 1:5 
- . G3*. 



.02 7 



• . (5 ^ 
.3^3 



310 



. ci: 



.-.m ::if> --Ah -.m -'.Sil 



0^3 
Oa b 
o55 



- .36^ 

- . u 1 6 
.097 



018 

-.027 
-.055 
.0 7«4 



.C 11 



'.063 
.1 Sk 



-.coo 

'Mi 

.02*4 
-. CC7 



.007 
- . o u 5 

- . u lo 
.G 7*4 



.3*42 



, 0 21 

.0 19 

-.165 
^ « c 



-.010 



C13 
.077 

-.140 

1 •> ft 



, 036 



0*43 



.0 16 

.056 
.09f 



50 



51 
5 3 

55 
56 
57 
58 



1:5 



.02 7 -.OIU 



.016 



.X 11 -.ceo 



m -'.m --At) -.m -:2?r-^l? 



15 

. 133 

' . Oul 
117 



. 215 
. 127 
■ . 13'» 
• . Ill 
• 157 
. 0 55 
. 139 



.3*3 
• ;*4 6 

. 022 
• .C67 



. fid 1 
.3^5 
. 119 

. 03 1 
■.039 

■ .CS^ 



, 0^3 
. 05^0 . 



• loU 
.22 6 

• t ^1 

12^ 
022 
06 C 
03 7 
051 



.097 

• di3 

-.369 
-.:053 



• 1)71 
.162 
.0*47 
-.019 

Oil 
- .066 
-•032 



0 27 
-.055 
.0 7*4 

2^0'*2 
.03*4 



• j&8 
.3 98 
.03C 
-.0 17 
-.062 
-.002 
-.0 98 
-.012 



.cut 

■.£.63 

.1 3*4 

.ul4 
■.C 25 
• 001 



.C3*4 

.ii 16 
•.I 2<* 

.0 
• . C *♦ 0 

.CC5 
. .G 7^ 

.w07 



.02*4 

CC7 

.:-8gr 

.0*4^ 

.005 



..C05 
•.C?2 
-.020 
.02*4 
-.015 
. 002 
.035 
.020 



.007 
- . u' 1 G 

.u 7*4 

• . 0*46 
-. jll 
. 05A 
.0-02 



-.355 
-.119 
■■.0,36 
'.001 
.0 8C 
.0 16 
.0 97 
-.002 



.3*42 

.t) 21 

.D'19 
-.165 
.015 
.02<* 
.018 

-.I3t 
-.11 8 
-.009 
-.311 
.081 
-.i J** 
. 139 
.0(37 



-.010 

-. C13 

. 077 

-.1*40 

• 028 
.029 
-.002 



-.110 
-.188 
-.028 
-.020 
.096 
. 002 

• 101 

• 026 



. 036 



.020 



"•0 le 

'.m 

»09f 

:3II 

► 0<i5 
.CIO 

► 22 9 

► 6*40 

•Mi 

► 155 
»01(« 
.111 
.012 



^^ SUHMARV. OF- -ir srATXSTlCS FOR ITEHS 



IT en 

I 
2 
3 
*• 

5 



8 

9 
lU 
11 

12 , 

13 * 

1*4 

15 
16 
17 

1 8 
19 
2D 
21 
22 
23 
2W 
€5 

2 7 
28 
29 

3^? 

32 
33 
3** 

J 5 



AVERAr,t 
RiSlDJ^L 
-.026 
-.C12 
-.035 

.on 

.337 

• 039 
.085 

..JO'S 

-.317 

-.u59 

-.?5**. 

-.0.55 

' -.3<4t» 

-.332 

- . n 3 2 

.o3H 

^:fib 

- . J 1 1 

- • n J 9 
.Cl^ 

- . i 

• 059 
.3^7 

- . l» 3 5 

.;^5 

-. J j I , 
.0)5 
.0)1 
j9 
.{Jl2 
.CO 

-.091 



AVERAGE ABSOLUTE 
RESIDUAL 
.0**6 
.022 
. 0 16 
. D 86 

. 3 $*♦ 
.C57 
. lUU 

. )66 

. J *4*4 

.0 75 
* J71 
,073 
. 356 
.D**5 
.0*47 
. J66 
.0^7 
.^73 
.093 
.091 
. u2l 
. ^62 
.L'58 
.0 36 
. 389 
. J5g 
. 1 3i 
.C39 
.0 75 

. J 2*4 

.060 
.0 3fi 
. 0 1° 

. 3 9r5 

. 239 



ROOT MEAN 
SQUARE RcSinUAL 

• 4 

^ .0*4^ 

.0 29 

.0.57 
.0 53 
. 181 

• a90 

• 359 

• 160 

• 1 29 

• 1 28 

•lie 

.0714 

• 065 

• 0 97 

• C92 
.u85 

• 111 

• 103 
030 

• 083 

• 061 
. 0 U 3 

• 125 

• 1 1^ 

• 153 

. 032 

• 1 oe 

. Ctt6 
.026 

.1 2*4 . 

• 301 



WEIGHTEP AVERAGE 
RfSIOUAL 
-. 0 CI 

.oco 

. OC 1 
. 001 

- . 0 CO 
. Oul 
. OGO 

- . (J CI 
• 001 

- . u CO 
. 001 
. QGl 
. 001 
.OCl 

. OijO 
. 0 CO 
, -.GCO 
. uCl 
. 000 
.001 
. 001 
. 000 
.001 
. GGl 
. O^Jl 

- . 0 CO 
. > 0 0 
. OCl 
. 0 uO 
. CCl 

- . a CO 

. OLl 
. 0 L' 1 
. 001 ^ 
-.OCl 
. Oi^2 



XEIGHTCQ ABSOLUTE ■ 
AVERAGE RESIDUAL 
.035 
.012 
.011 ' 
.105 

- . .055 
.037 
.037 
^ .099. 
.0**7 

.0.*4l 

.0 21 
.02*« 
.029 
.019 

• Old 
.020 
.0*43 
.018 

. 078 ' 
.106 
.09*4 
.fill 
.068 
. tl36 
. 035 
.059 

• 013 

• 096 
*02*4 

• 0*«'8 

• 021 
• •051 

• 029 

• Oil 

• 0 60 



\ 



in 



27 

kZ 

kb 
k7 

. ^ A 
*.9 
5J 
51 

12 

55 
56 
57 
•58 

AVERAGES 



• Gi 

-.512 
.}U2 

.:i2 
- . 0 j ^ 

• w • L 

-.C19 
. J 2 n 

-.03 J 
.029 
.(,'37 

. j j 1 



.052 

. D^a 

.0 22 
. G3U 
.C53 
. 0S7 
. 0 65 
• u91 
. C 35 
.0 37 
. 0 31 
.0 66 
. 20C 
.C^«5 
.3^6 
.0 62 
.026 
.C7^4 
.03^4 

. J65 



.0 

. 059 
.063 
.033 
. 0^49 
.057 
.0 39 
. 0 61 
.113 
. 0*47 
. 3<«U 

.0*4?. 

.0 99 
.25^ 
.05^ 
. C62 
.071 
. C50 
.076 
. 051 

.0 86 



V 



. 001 

::8£8 

. aoo 
. oci 

0 CO 

• . G ci 

oco 

. OCI 
. 001 
. Oil 
.010 
. 001 

0 CI 
. OCI 
. OCI 

0 CI 

-.on 

. OCI 
0 CO 
. 0 Gl 
-. 0 CO 

.OCO 



.029 

'M 

.0^8 

:m 

.020 
.03^ 
.032 
.048 
.070 
.015 
.040 

:m 

.101 

:m 

.051 
.009 
.074 
. 017 

.044 



I 



< SUMKARY OF FIT S T A T IS T ICS. F CR ABILITY LEVELS 
ABILITY LEVEL (.y'6-PCINTS» 



-.001 

.059 

.099 



, JCl AVERAGE ABSOLUTE RESJDUaL= .065 ROOT WEAK SQUAPE KESICUALr 



/FIT STATIST ffc 
'^AVfRAGc rESIDUVL 




^ 1 
' -2.75 
C 18 


2 

-2.25 
. 039 , 


3 

-1.75 , 
.00 4 


-1.25 . 
-.00 2 


5 

-.7 5 
-.00 6 


6 

-.25 
.00 2 


7 

.25 
.005 


8 

.75 

.002 


9 

1.25 
-.005 


10 

1.75 
-.00 2 


AVERAGE ARSoLUT£ 


RESIDUAL 


* 

.120 


.103 


.U93 


.0 61' 


.C 46 


.u2B 


.029 


- .042 




^050 - 


* ' RCOT .fEAN SaUA^-. 


RESIDUAL 


.17 6 


.135 


. 112 




.13 65 


• .036 


.036 


.059 


.0 71; 


• 0 62 



























12 

2.75 
*.0Q0 

• 096 

,190 



.0 94 



' o . 213 
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STANDARDIZED RESlOUALS- 1 PARAHETEf? J^CCEL 



MID-PCINT 
NO. OP 

ITEM 
I 
2 
3 
u 
5 
6 
7 
6 
9 
10 



11 
12 
13 
lU 
15 
16 
17 

le 

19 



I 

-2 . 75: 
Ik 



• 1:26 

- . <«5(/ 
. 166 

- . e37 

211 

1. 311 

2. 700 
5. 36J 

- . 1*67 
-.001 



•4. 313 
•2. €63 

►1. ;5 7 

-2. 

3. 271 
- . 376 

C7 3 
.0 5^ 



SI 5 
. 339 

- • 5i. t> 

I . 6u9 
31U 

H . 313 
376 

- . 7u 4 



2 3 *• 

-2.250 -1 .75 C -1.250 

5U 91 22U 



•3.065 
.0<«6 

-.955 

- .68 3 
-1.07.3 

C.92S 

2.292. 

6. <9 8 

7.233 
-1.596 - 



2. 5:1 5 

3. lie 

1. 969 

2. 51U 
'2. ^3 1 

6. 219 

3. U91 
5. 571* 
3. U21 
2. 6U5 



•3.256 
•<#.126 
■3.075 

■ 1.737 
•1.967 
2.199 
-.739 
-2. i96 
• 2.196 



•1.918 
.66*^ 
^.289 
U.17 9 

-1. tk5 
6.75 J 
- .739 
6,7U3 
. U7 
2.188 



-. 95 6 
-2. 067 
-2. 668 

-. 63<« 
-2. i93 

-.597 

2.U52 



-3. 



^2 

7J 3 



■2. 9U1 

2. 150 

3. 9'*e 

5.678 

t*. 075 
. U95 
6. 786 
68U 
2. 777 



-2.213 
.235 
-.292 

.5.01<« 

-2.313 
3. 85? 
3.21w^ 
5.2U6 
1. 390 

-3.205 



-.657 

".in 

.100 

•1.523 
969 
-5. 825 



-6.185 
-.196 
7. 79U 
<4 • 18 u 

-3.577 
1.66 7 
-.UU7 

1.913 



5 

-.75? 
32 5 



..<«66 
• 046 
.3 73 
.5,5U3 
•3.636 
.857 
.81U 

3.307 
.8U0 

-•928 



1.^^665 
1.U05 

.9ua 

.253 
.388 
.<«fl2 
-.0 8 5 
"1 . 05 3 
•U.26g 
•7.0 82 

•b.0'91 
-. 561 
3.7U2 
.160 
-1.893 
1.686 
.Ull 
6.^13 
.3<«9 
-U. 03 1 



6 

250 
5C 3 



2 .566 

1.079 
• 663 
-3.381 

-.U75 
•1.507 

-.5U1 
-2.197 

3.<«39 
.5«*0 



2.0 18 
2.205 
2.5€4 
1.059 
.571 
2.296 
2.675 
-.939 
2.809 
- .59U 



-.ue5 

-.2U6 

- .766 
1.206 
•Z.528 
.229 

A:m 

•2.1 51 



ABILITY LEVEL 
7 8 



.250 
U67 

1.757 
l.OUi* 
1.810 

•1.57U 
•2.116 
•U.5 10 
*.532 
3.C87 



2. U.J 8 
1.760 
3.309 
1.78*« 
2.786 
-.215 
•3.171 
1.656 
2. 97 8 
<».U95 



2.860 
'1.6U6 
•2.612 
•1.117 

1.5U3 
•3.272 

-*096 

l.<«8 7 
-2.11 3 
.62 2 



. 75 0 

33 9 



9 10 
1. 25J 1.750 
2U9 102 



2. 



-.Ill 



• 997 
.161 
3.522 
•1.096 
•1.556 
•5.603 



.951 

1.529 
1 . 061 
1.97<« 
-.0 30 
-1.822 
-l.U 78 
-.2U7 
2.1 37 
6.8 65 



5.957 
.351 
•3.8U9 
-1.518 
.0 86 
-.937 
•1.633 
-2.8 83 
l.<« 26 
.5 89 



-.113 
-.11<« 
.289 

e.ou2 

2.187 
-.25k 
-.193 



.931 

.289 
-.126 
1.168 

.2 61 
• 1.385 
•2.6U3 

.587 
2.356 
U.039 



i4.U5l 
-.995 

•2.197 

•1.232 
1.380 

•3.531 
.67 8 

•6.833 
1.6UU 
2.0r8 



1.279 
1.015- 

mt 

1.363 
.031 
1. 04*6 
•2.005 



1.015 
1.015 
1.015 
1.015 
1. 0U9 

J!?10 
1.111 

3. 251 
.020 

-. 560 
.961 
. 838 

-.017 

-.557 
•U. 2U9 



11 

2.25J 
Hk 



-. 8<«8 
.667 
.667 

1.176 
.<«80 

1. U20 

1.935 
.175 



3 - 



12 
2.750 

3 



.17<« 
.17k 
*17U 
.351 
.25<» 
• 53<* 

.2ie 

.17k 



. bb7 
.667 
. 667 
.667 

0. 667 
..667 

1.019 
•1.18U 
1.382 

1. k7b 



1.627 
.667 
1.371 
1.728 
.610 
1.007 
2.1U7 
-5.180 
.9C3 
2. 1(0 



.17k 
.17^1 
.Ilk 
.17U 
.i7k 
.17k 
. 201 
1.1<«5 
.272 
. 291 



. 321 
.17*. 
.527 
.581 
.355 
.198 
1.375 
-.665 
.932 
* 2. lUO 



.. 72 X 
1 . 25 b 
- . 652 
. 610 
k. 251 
I*. 996 
-. 559 
-I . 755 
>. 7g9 



. 167 

1. klS 
1 *klO 

.^91 

7 . C8 2 

2. 116 
-3 . '5^ 

•2.336 



31 6 

1. 851 

2 . 70 X 
1. 23t 
k. 321 
9. kit 

3. 9C k 
.5, 5C 6 

1. ^kZ 
-2. 370 



-1 .968 -.9t»l 
-A.eo5 -2.192 



l.G U2 
•1.03 3 
2,886 



'2.r - 

2.572 
1.315 
.953 



13.225 12.252 
2.55^ 1.772 

-2.265 



•1.688 



-.153 
•2.903 
.102 
-.071 
•2.7 16 
5.626 
- .2Uf 
2.906 

1 .6ue 

-.968 



l.*»53 
-.C69 
-.71U 
1.522 
3.27q 
.Skk 
..9C6 
3.360 
2.250 
2.551 



1.185 

i.Tkb 



-2 



.0 56 



- 21^ 
-1.953 

-5 .7 32 
-.5 57 
3.U 68 

-5.1 98 
3 .060 



.509 
3.685 
•3.377 
.911 
.2 97 
•7.536 
'i. U53 
2.9 16 
•5. 081 
l.^kk 



1.9U0 
2.<*23 
-.371 
1.015 
.762 
-8. 022 
-. 83fl 
.1. Ul 

• <*.3ie 

2. 103 



-. 1 CO 
.668 

-.163 
.667 

1.227 
•5. 810 

-.826 
.6U8 



. 190 
'3.073 
. k50 
.17k 
.2k2 
3.626 
.525 
.17U 
*k.k23 
.2kl 
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ANaLV^IS OF STANDAR0I7E0 RESlDUALs 
<A330LIT£ VALUES) 
INTERVAL NJHBER PERCENT ^p^R^^jSiT 

( TO 1 2 77 39. 50 39. 8C 

: TO 2 16i) 22. 99 62. 79 

Z TO 3 117 16.51 79. 6U 

3EY0N0 3 Iti'd 20. UO lUJ.UO 



sImhARY of Fn STATISTICS FOR ITEMS 



IKM AVER43E 
RcSlOJAL 

1 - . 1 

.117 

.05. 

L . 0 > a 

.677 
.675 
.2j. 
. 5 
.302 
. 029 

.0 JD 

. l'*2 



5 
5 
-7 
8 

1? 

11 
12 
13 
l** 

15 
16 
I 7 



1 



AVERAGE ARSOLUTE 
RESinUtL 
1. ^6^ 
.723 
.781 
3. 112 
I. 739 
1.-797 
1.697 
3./502 
2.126 
1.596 
1. 5^** 
1.528 
1. 651 
1.^5. 



89 
126 
859 



ROOT MEAN 
SQUARE RESIDUAL 



8 10 
1.063 
.9*41 
3.7 26 
2.117 
2 . 22 3 

1 . 7S3 
(4. 331 

2 . 7^2 
1.96; 
1. 967 

1 .91*4 

2.&13 
1.50^ 
1.307 
1 . 33C 
2.195 



WEIGHTED -AVE RAGE 
RESIDUAL 
.831 
. U76 
. f^^2 
. 11 
.'•31 
• 012 
-.07(4 



323 
. U58 
. 87 0 
.200 
. 115 
.207 
. 882 
.629 

. 1'46 

.^59 



WcIGHTeD ABSOLL'TE 
AVERAGE RESIDUAL 
1 .7*48 
. 7uQ 
. 888 
^.181 
2.273 
1 .606 

1 .'♦'48 

3. 990 
2.061 
1 . 987 
1 .590 

l.«49l 

1.807 
1.223 
. 960 
1.182 
1.976 
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k 



19 
20 
?l 
12 
25 
2«» 
25 

23 

27 
2« 

ri 

32 

II 

35 

3 7 
38 
39 

<♦! 

<*5 
W6 
«•/ 

«»9 

5 U 

IJ 

«3 



.56 
57 
56 



AVc RAGcS 



- . 2 2 o 

• 1 o 

- . 3 } ? 
, 0 b J 

i .33 9 

-.078 

1.300 
.329 

.22^* 
. <»8 3 

• 25f, 

.mi 

1.038 - 

-.063 
-.757 
.071 
.2^5 
.23J 
.2) 
-.859 
-.<»25 
w 81& 
1.8j 3 
. 91W 

-.013 

5^ •» 
-.u29 
-.395 
-.139 

.0 66 
-.026 

.IS8 

.277 



. 9/b 

3.08'* 
. 66a 
2. a<*5 
I. 88i* 
1. 15<« 
2. 318 - 
. 89? 

601 
. 92l 
I.92L 

• 80^ 
2.056 
1.13^ 

. 865 

iJll 

I. 3 55 

1. 7 22 
1. 163 
.602 

• 866 
1.577 

1. 161 

2. JU6 

1. 201 
. 775 

3. 2«»9 
5.026 
1. 372 
1 . 19{ 

1. 832 

2. W79 

• 83^ 

1. 9 l.i 



Fir STATISTIC 

a\,er/ge residual 

Average abscluth resioual 

RCOr ^EAN SnUAt?- RESIOL'AL 



I 

-2.7 5 
. 5W3 



1 .63 C 

2 . J7o 



I. J 33 

!r'S§9 

I .716 
.9 06 
3 . 251^ 
2.3^2 
1. '♦79 
2.9 15 
i . 121 
t».9<*6 
1.085 
2.092 
1.017 
2.23^* 
1 .5 53, 
1.0 32 



;695 

\7i 



5^6 
_ 6"37 
2.95** 
3.505 
1.9 22 
1 .<*91 
.8 29 
.973 
1.697 
i .279 
2.128 
772 
l.9<*7 
1. 391 
.96? 
3 .W97 
5 .<406 
1.515 

1 .390 
2.1*45 

.7 19 

2 .858 

,9<4i4 

2.218 



-.170 

.563 
.591 

. 361 

.253 

.073 

• 118 
-.918 
-.100 
t. 770 
-.3 35 
-.1<*1 

.235 
'^..020 

.106 
•.2<4l 
-.685 
2.69 9 

it65 
1« 230 
-.750 
•.<*06 
-. 39'» 

.059 

.165 
-'.21'* 

.158 
-.377 
2.232 
-•0<*9 

.072 

. 26<« 
1.2D<^ 
-.566 
-.7 C7 

.322 
-.7 C6 

.087 
-.657 

. 2C0 

.23 3 



i 



1.027 

3.665 
.726 
2.8^1 
1.509 
l.-,387 
2 .387 

• 560 
<♦ .520 
1.200 
1 . fl57 

.945 
2.0<«0 
.109 
..000 
2.669 
6 .7<«2 
1.178 
3.03<« 
3.121 
1.981 
1.190 
.583 

• 83( 
l.<*19 

^l3o5 
<».522 

. 915 
1.58^* 

. 710 
3.056 
3.7<*7 
1.5<42 
1 .160 
2.325 

.339 
3. 261 

.671 

1.977 



SL'MMARY OF FIT STATISTICS FCR AGILITY LEVELS 
ABILITY LEVEL<MIO-POINTS> 



2 

-2. 25 
- 8*4 2 



2.562 

193 



-1. 75 

.66t' 



2.736 
3. 28<4 



-1.25 
.327 



2.6'*2 
3.716 



5 

- .7 5 
. 329 



2.20U 
3.195 



6 

-.25 
.25*4 



1 .63<» 
2.u<42 



7 

.25 
.398 



1. 681 

1. 979 



I 



H 
I 



8 

.75 

, 170 



2.0 76 
2. 699 



9 

1 

-:096 



2.0 7<4 
2.771 



10 

1.75 
. 113 



l.€<4€ 
2.231 



11 

2.25 
.155 



1.3<46 
1. 863 



^.75 
•.07<4 



.693 
1 .355 



ERIC 



223 



2?A 



EX?ECTeC P VALU€S-3 PARAMETER MOb£L 



ABILITY Lev EL 
6 ^ 



CAT£GO^Y 

MIO-PCINT 

NO. OP 

EX HINSSS 




ERLC 



227 



nnSEfcVEC P PARAMETER MCOEL 



CATEG (RY 



ABILITY LEVEL 
7 8 ^ 9 IC 




MI O-PCINT 
NO . OF 



ITEM 
L 
2 
3 



» 6 
7 
0 

10 



250 
, iuz 

79Z 
< Ou 0 

125 

30 3 
. 167 
, 75Q 

► Zti\~ 

> 12 5 



• 6 iw- 

. cac 

. l*«u 

• 1 «0 

• 16(7 

• .€^J 

. 2 ft J 



• V«* 7 
. ^19 
. 31 6 

• 0<«<« 
.167 
.2^6 

.2«*e 

• 6u 5 
.1^9 

• 366 



. 613 
. 635 
.897 
. 077 
. 2^7 
. 232 
.237 
. 603 
.113 
T5C5-- 



.786 
. 90 
. 962 
. 1<*6 
.37<t 
.16 2 
.252 
.626 
.162 
.696 



. 907 
. 9^6 
. 961 
. 3^*5 
. 5a<* 
. 223 

\m 

.259 
.827 



. 9^7 
.q76 
. ?S0 
.652 
. 786 
. 3V6 
. 361 
. 676 
.295 
.923 



.981 
.966 
.995 
.699 
.913 
.<*67 
• S03 
.607 
.353 
.97 6 



2i*8 



. 96 0 
. 96 6 
.99 2 
.966 
. 952 
.6i* 5 
• 70 2 
.76 6 
. 52 0 
.99 6 



.-*75 
90 



1.000 
1. CO 0 

1. 203 
. 978 

^ au9 

. €67 

. «ill 
. €56 
. <69 



11 

2.25 
32 



12 

2. ?5 
11 



.^69 l.QCO 



U 
0 00 



CO l.OCO 

OQIi 1.000 

.909 

675 1.000 

.969 1.000 

.936 I. 00 0 

.719 .909 

irOOO 1. 000 



12 
I 3 

l^ 

15 
16 
17 
Id 
19 
23 



26 

\l 

29 
3J 



31 
.^2 
33 

35 
lb 
3/ 
3d 
39 



1 67 

> t^l 
061 
, 125 
. 03 3 
. 16 7 
. 5jD 
. 000 
» ««D 4 
. 00 Q 



.000 
. 70 8 
. 25J 
. 20ft 

► ce3 

» 5«3 

. 20e. 

. 000 
. 250 



. 250 
. 333 
. 063 
. 956 
. 667 
. 253 
. 125 

: • J 

. 292 

003 



, 6 CO 
,3*0 

> 380 

. i^ec 

I 3 

> 3^0 
.3 00 

. oco 



0 2J 

> 7 eO 

. e2Q 

. 003 

> 1 cC 



, 3**0 

• 3 to 

16a 

• 8*»5. 
, 560 

, 160 
, 2 CO 
.163 
, 380 
i 23 



, 83 7 
, 607 

► 63 2 

► 73 7 
,6l«* 
^623 
»«*7i* 

. uo c 

.053 



.03 5 
. 66 6 

:Wz 

.07 0 
. 53 5 
.026 
. 18 <♦ 
.009 
. 21 9 



,33V 

► 2«*6 
. 193 

► ft77 

► ^♦12 
, 23 7 

► 132 

► 15 8 

► ^♦3 3 



• 9'«3 
. 692 
. 692 
. 79<» 
. 773 
.78U 
. 5u5 

.057 



. U^l 
. 856 

► 26 9^ 
.196*^ 
, 1*»9 

> 531 
, C15 
»186 
.031 - 
.16C 



. 97 5 
. 96? 
.92 5 
, 926 
.655 
. 66<« 
.61 3* 
.031 
. 311 
.166 



.1^5 
.906 
.2**2, 
. Ik3 
.296 
.66^ 
I 0^^ 
' 1^2 
,063 
.169 



.'♦12 .597 

.222 .321 

\^,\ 

.^♦33 .5*»*» 

. 253 . 27i» 

.196 .192 

. 387 . 6if8 

.^'♦l .6'*5 

.29*» .<i53 



. 96*» 
. 975 
. cft't 
. 966 
. 9<«1 
.9 3«* 
. 759 

.^77 



939 
. 3 (0 
.2 25 
, i»57 
. 652 
, 0 5C 
. 193 
. 1 C5 
.3 02 



. 732 
. 

. 330 
. 966 
.582 
.230 
. 28*4 
. ft30 
. 711 
.6 CO 



. 996 

.933 

. 7i»5 
. 132 
. 782 
. 619 



. 7*»1 
. 962 
. 29? 
. 310 
* 599 
. 756 
. 061 
.2Ci» 
.130 
. '♦8 5 



. 659 
. 65i* 
. '♦'♦ft 
. 972 
> 666 
. 196 
. 365 
. 92 5 
. 7ii5 
. 762 



.997 
.995 
.96 9 

.992 
• 965 

.651 
.196 
.661 
.959 



.921 
.976 

.^♦10 

.66*f 
.101 
.250 
.291 
.67*» 



.913 
.837 
.590 
.976 

\\n 

.51<^ 

• 98 1 
.76 1 

• 908 



. 996 t« GCO 
.996 1. COO 
.992 1. 000 

.966 l.^OOJ 



1.000 
1.0 00 



i.OOO 

1 . C 0 Oo 



.97 2 
. 895 
, 327 
. 9<^0 
.98 0 



»00Q 
Sll 
, i»8 9 
969 
COO 



.97 2 1. 000 
.96 6 '•9fl9_ 



59 3 
,565 
, 66 3 
. 67 1 
.25i^ 
.230 
, 7 
, 66 7 



.95 6 

.99 2 
. 667 
.153 
.60 1 

.99 2 
.631 
.93 5 



^B9 
769 

c22 
^56 
333 
^♦11 

567 

90 0 



> Ct.. 0 
. 000 
. 789 

> OOU 
. 956 
. 17 6 
, 733 
. 003 
. 833 
.989 



l.QQQ 
. 969 

.<#g6 

1 . 0 00. 
l.QQQ 

I.QOO 
l.OOC 
1^000 
.969 
c .969 
l.QOQ 

'M\ 

.750 
1.000 



• 727 
l.QQO 
1.000 



1.000 
1^. 0 0 0 
1.000 
1.000 
l.UOO 
1. 000 

• 616 

• 273 
.616 

1. 000 



" .969 l.OCO 
.9 69 .qC9 
.936 1.000 

.313 .091 
.615 i.nOO 

1.000 1.000 
.875 .909 

1^000 1^000 



I — 
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22J 



'4 3 

50 



J • U 0 0 

G « iipa 
i • iju 0 
.157 

. . too 

. 333 



.2*0 

• u ao 

• 060 

. 0 to 

• OAO 
. l.K 
. 130 

• ^ <u 



.053 
. 2«*6 

• 36a 
.167 
. 06l 

. l^3 

. 12 3 

• 0 S 8 

• 123 
.'♦21 



^t5 2 
, 19< 
► 53 6^ 
, 397 
. 10 6 
,067 
. IZU 
» u 3 1 

, 675 



. 06u 
. 51 3 
.651 
. 538 
• liU 
.585 
. 12 3 
.03r 
. 239 
. 783 



. €52 
. 770 
. 718 
. <«23 
. 166 
. 095 
.06^ 

• 362 

• 66 



. 269 
. 776 
. 857 
. 7 9i» 
. 5U9 
. 3<«6 
. 106 
. UQ 
. 568 
. 910 



..3 7C 
.875 
.9<»6 
.6<«5 
.63j3 
.5'»3 
• 

.190 
• 761* 
.951 



.^621 
. 96 
. 96^ 
.87 1 
. 78 2 

. 7k e 

. 073 
. 33 1 
. 81 9 
.99 2 



. €22 
.93 3 
. 97 6 

. 856 
. 222 
. 533 
. 911 



969 1.000 



.000 
. 969 
.906 
.906 
,969 
.0 63 
.'♦69 



1. 000 
.618 
61 6 



>906 1.(jC0 



.978 1. 000 l.OQJ 



51 
52 
53 

55 
56 
57 
58 



167 

5*t2 

. UOO 
12 5 



. 1<4C 
,560 
5'tO 
100 
U CJ 
2 20 
0 00 
2C0 



.123 
. 395 
.58,8 
.&5<* 
.c02 6 
.351 
.026 
. 333 



. 129 
.515 
. 686 

.0C5 
.5 21 
.010 
. '♦53 



. li»2 

•hi 

. 59«» 
. 036 
.55 0 
. Oi* 7 
. 63 2 



. 1«^5 
.6C5 
.6^0 
. 766 
. 1J5 
. 686 
• 1 16 
. 76 6 



.153 
. 611 
.862 
. 83 5 
. 22<« 
.617 
. 326 
. 825 



.196 
.671* 
.905 

♦ 699 

• ^♦27 
.697 
.560 
.899 



.226 
.730 
. 9'»'» 
.90 7 
. 63 9 
• 93 1 



. <4ll . .686 • 5<«5 

.733 .625 .727 

. 967 , 969 l.OQO 

.9<»<» .969 l.CQG 

,769 . 875 l.OQO 

• 9<»(» 1. 000 1. 000 

. i^^k .938 i.qoo 

.969 i.doo i.ioo 



230. 



I 

I 



r 
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RESIDUALS- 3 PARAMfcTER f'OOEL 
(OOsERVtO-tXPECTEOI 



HI O-POINT 
NO. OF 

ex MiK£:e:5 

I TEM 
1 

I 
3 

^ . 
5- 
o 
7 

a 

9 
IG 



I 

2<* 



I 056 
09 3 
20<* 

, M 
til2 
U91 

. a^^ 

. 26 3 
, 095 
. C26 



2 

250 
5U 



. C63 

• lU^ 

• 036 
. ijl5 

■ . 036 
. (31 
.129 
.001 
.071 



-1 . 75C 
liU 



. 012 

Ou 

.* 8fe6 
. 032 
. 05i} 
. 020 
. 0<«^ 



•1. 25u 
19<* 



>.00d 

>. ji;3 

JC7 
*012 
.015 
. 0^6 
.Old 
.003 
>. J3^ 
.01)3 



5 

750 
31 tt 



.00 0 
•.002 
.3 15 
.015 
•.'J06 
-.01& 
.017 
-.016 
• 006 
.001 



6 

-.250 

.0 13 
-.CC2 
- .U 10 

' .1- 16 
,0 1€ 

•lb 35 
• 03<« 

-.0 16 



ABILITV LEVEL 
7 6 



.251 

50'3 



-.n 
.01)3 
.005 

-.019 

..Ou3 
.025 

-:m 

-.002 
-.007 



. 75C 

36 6 



.003 
.OOt 
.005 
.003 
.0 C6 
-.006 
0 27 
.0^6 

• 006 



9 

1. 250 
2^6 



310 
032 

2 

,6 
010 

■M 

• 0 <i8 
«0Q2 
.308 



10 
1.75C 

90 

.010 
.010 

'Ml 

-.006 

«r8?5 

.067 



11 

2.250 
32 



• 0 21 

:oio 

• 0 10 

• 010 
.0 10 

-.01.9 

. o2i 

.0 67 
-.036 

• 010 



12 
2.750 
11 

«010 

•Pi 
.0 10 

-.061 

-.061 

• itZl 

.020 

.12€ 

.065 

.010 



11 
12 
13 
!<♦ 
15 
16 
17 
16 
19 
20 



<3 
2V 

25 

^6 

? 7 
^8 
c9 
30 



31 

J^ 

23 

35 
36 
3 7 
38 
39 
^1 
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. 1€6 

- . 2<*6 

■■•All 

-.196 
. 23 2 

*.ai2 



086 

, 123 
, 039 
, 181 

036 
Oj 9 

. 116 
0 71 

, 119 



0^*6 



.037 


. 021 


.027 


- • 1<*0 


, 1j6 


.026 




Oil 


.062 


■■AU 


. C^B 3 


- .022 


- . 152 


. »}3^ 


.C33 
.0^1 


- . 132 


. GIC- 


- .066 


. 026 


-.031 


- .Ul<* 


C18 


-.016 
- .0<*9 


-.03 9 


D9 0 


.0u3 


. 0. 2 


-.Oil 



. C16 


.QuD 


. uu 9 


. C3<*- 


. C23 


. 0^3 


. ID 2 


- . 15 2 


-•015 
. 


. 052 


-.03 7 


. C31 


- . 16'* 


C77 


. :20 
. 022 




\m 


. C50 


-. (<*1 


. uld 


. 017 


.021 


-.bl5 


. 10 2 


. G3 J 


. G63 



i07l 



• 0 J L 



-.012 
'.022 
.035 
. C31 
>. Z^^ 
'.0^3 
-.311 
.011 
•.0C2 
-.013 



.125 


. O'w 3 


U23 


.088 


. 029 


-.312 


. (<45 

.f.C2 


. 035 
OL fl 


'--AD 


. I**** 




- . 03 C 


- .05J 


-:m 


■ .0^1 


. 075 




. 029 




c^l 


.Ulf^ 






- .0u9 


-..U8 


066 


.025 



01^ 
0 02 
, Q m 
.013 
009 
0 28 
012 



. 0 07 
.010 
. 0 H 
.03^ 
,0 16 
,017 
.012 
.010 
.011 
.0 2<4 



.0 05 
.035 
.006 
>.002 
'.005 
.062 
'.018 
. 0 2« 
.02a 
. 3 33 



•.0C9 -.021 



-.006 

• .C 05 
.0 10 
.003 

.n 13 

.0 25 
.O'^S 

-.002 
.169 

. .002 



.0 26 

.u 33 
«0 12 
.0 39 
63 
.005 

■■Ml 

' .C 0^ 



. 006 

-.00 6 
-.002 
-.010 
.002 
-.0 16 

■•Td 

.01^ 
- . CO 1 



-.006 
.000 

-•oGe 

• 021 
.006 

••0 17 
.036 
-.0 C9 
-.027 

• GIO 



.0 07 

• 0G5 
-.0 01 

-Til 

-.0 2*4 

• 0 07 

-•ooc 

- .Cil 
.001 



-.0 06 
.0 03 
. 003 
-.306 
' .0 37 
.0 37 

. . ca6 

.0 12 
.020 
-.012 



-.1-09 


.007 


-.008 


-.OQC 


•CQ^ 


-.01^ 


- .u Oft 


- . uu 1 
- . 0 U 2 


.01^ 


.coX 


-.00<4 


-.t 52 


-. na6 


.0 26 
-.019 


.0 18 


-.016 


.U07 


-.009 


.0 18 

.0 01 


-.003 


-.0 


.C31 


.006 


-. 030 


-.1 1«» 


-.Ol5 


.0 09 



.0'J6 
.036 
.002 
.010 
-.002 
-.013 

-.023 
-.010 



-Ml 

-. 0 22 
-.3^3 
-.037 
.002 
.020 
-.0^1 
-. 00^ 
.017 

-.00^ 
.001 

-.0 ^2. 
.33 ^ 
.0<«6 

-.059 

-.026 
:002 
00^ 

-.020 



.00^ .0 15 -.0 «»5 -.01^ 



. 0 10 
. 010, 
. OlOl 
.CIO 
. CIO 

■■m 

. C02 
.010 



.010 
-. (00 
-. C^l 

::m 

.G5<4 
r65 

.096 
C^«» 
C38 

-.m 

-.013 

:m 

-.035 

-.m 

C3 6 
.008 

.008 



.010 
.010 
.010 

• 010 
.010 

• 010 

• 036' 
-.039 

• 010 

• OIC 



.010 

• 010 

• 057 
.057 

-•0 CI 
.073 
.031 
-<.05<i 
-.015 
.02^ 



• 061 

.010 

.061 

.0 96 
-•025 

.010 
>. 026 

.010 ^ 



,051 



• 010 
.010 
.017 
.036 

• QIU 
.053 

• 05? 
-.15€ 
-.0'^i3 

• oa: 



.010 

'•061 
,C7i« 
• 016 
.057 

..li«7 

\Vd 
-.015 
.010 



.03<« 



23i 



V 3 

50 



53 
5U 
55 
56 
57 
58 



• My? 

ll<4 

. 067 
> . 063 
••0 42 
' . C7 0 

. Cfi9 



. 037 

• 136 

• 067 

• 130 

:m 



Ml 



• OdV 
(iOO 
'•Cj5 
.057 
.CC 1 
. i5 3 



. 12 0 

>. 012 
■.C23 
-.110 
-.C3 5 



081 
04u 

Old 

043 
,014 

ra7 8 



o:8 
jdi 

C15 
06.* 

. oil 

. 021 



3 5 



. C 10 


- • J t i 


- .t 11 


-.ul W 


-.0 . 




-.031 




• (110 


- *. S ij 3 
-.071 

.319 

-•ai9 

C26 
• 033 


• U C 7 

.018 
-.025 
-.012 

.i)15 


.071 
-.006 
-.UlC 
.U05 
.001 
*002 


':m 

.CLl 
.0 31 
.001 

-.008 
.012 

-•C12 


.021 
-.010 

- .C '3 

- ♦ 0 u<i 

.033 
-.0 06 


■y.ni 

-•04C 

• 016 
-•039 

• 015 


-'Mi 

-. €29 
C53 
' .089 
.051 
-.020 
-.CIO 


-.034 
• 0 0 2 
-.139 
-.t91 

-Ml 


-.152.. 

• Oil 
.072 

• 01^ 

6014 
?010 


- . tib 3 
.u0 3 
.50 1 
.016 

-. n9 

.075 
-•CO 9 
-.024 


.0u7 
-.0 31 

• G02 

.310 
-.OC 8 
-.028 
003 

.027 


.U05 

•Sio 

.045 

.:jo.5 ^ 

• U26 


-.003 
-.013 
-.007 
.C02 
\ . .010 

^ -m 

-..021 


.006 
.015 
-.002 
-*?0 06 
.0 16 
.0 10 
• Oil 
-.015 


-•035 
.036 
• 0'39 
-.0<»1 
-.035 
-.033 
-.036 

.oa2 


. 022 > 

• 007 

• 012. 
-.028 
-.036 . 
-.019 
-.065 

.013 


.118 
-.132 

• COO 
-.0 17 

-An 

-.029 

• 015 


\-^202 

X.ose ' 

• wio 

• 0 3 0 <^ 

• oil 



SUMMARY or F IT 

ITEM AVERAGE 
RESIDUA L 

1 .0)2 

2 .015 

3 .OlJ 

4 - . C 5 5' 

5 -.OQt 

6 - • J J 3 

7 .0)1 

8 .038 

?• 01 :^ 
.009 

-..017 
-.0:9 



STATISTICS FOR. ITEMS 
AVEKASE <iBSOLUT£ RCCT 



11 
12 
13 
1 4 
15 
16 
17 

i§ 

20 
21 
22 
23 
24 
25 
26 
27 
28 
29 
3J 
31 
'3 2 
33 
34 
35 
36 



-•Ml 

-.018 

.'M 

-.023 

-•on 

• jj J 

• J n 
-•qo. 

• uO^ 
-.Old 

.J'f? 

-.03'^ 

- . L ] 3 
.014 
. jL6 

• 013 
.011 
.CL6 
. Oo9 

- . iJ J 4 



• Q 16 ■ 


• 0 23 


• 019 


• 0 32 


• 0 33 


• 066 


• 321 


• 030 


• 015 


• 3 25 


• 0 36 


• 042 


• 322 


• QZt* 


• 077 


• 089 


• 029 


• 039 


.0 17 


• 025 


• 026 


• 0 51 


. 049 


• 087 


.3 27 


• 351 


• n29 


• 0 55 


• 0 32 


• 059 


.0^3 


• U65 




• U65 


• 0 32 


• 0 62. 


. 043 


.046 


.009 


• 013 


. J 11 


~ • 012 


. 0 1^* 


• 018 


. 0 25 


• 031 


.032 


• C34 


.0 31 


• 03& 


.067 


• 061 


• J 23 


• 029 


• 042 


• 060 


..j2'. ' 


• 023 


. J 29 


. 037 


• 027 


• 043 


• 0 36 


• 05C 


.0 29 


• 035 


. 0 21 


• O^^' 


. 091 


• 127 


. 0 5.. 


• 0 62 



MEAN 

RESIDUAL 



WEIGHTED AVERAGE 
RESIDUAL 

• 001 * 

• 002 

• 0 01 

• oor 

• OuO 

• 001 

• OO'* 
. UUO 

-.0 CI 
.001 . 
. 005 

. 104 

. 0 03. 

.003 

.001 

. 001 
-.0 CO 
^.OC? 
-.0 07 
-.0 CO 
-.000 

. OGO 

.002 

• OCl 
-•CC5 

• 001 
-. 0 (1 

.000 
0 CI 
.001 
.001 
.005 

-• oco 

. OCl 

• 000 
.0 02 



HEIOKTEO ABSOLUTE 
AVERAGE RESIDUAL 

• 008 

• C06 

• 012 

• 012 0 



J2*» 

• 019 

'An 

• oil 

• oil 

• 019 

• 012 

• 01^ 

• 014*' 

• 027 

• 029 

• 013 
.038 
.005 
.012 
.00 8 

:m 

.027 
.038 
.015 
.018 
.017 

.oir 

.012 
.016 

.gi<» 

.007 

.0^6 
.033 



ERLC 



23. 



i9 

(•2 
<»3 

<*a 

<»9 

51' 
52 
53 
5<# 
55 
56 
57 

5a 

AVERAGES 



-.052 



9 
C 2A 
0 2ft 

; j« 

059 
?26 

J 39 

3 25 



. J36 

• 0 5^ 

• 013 
. 0<»1 
. J2C 
.025 
.0 2.- 
.u2r 

• 0 33 



.035 
.3 75 
.055 
.070 
.0 57 
.335 
.057 
.0 62 
.029 
—0^-5- 
. 07U 
. 072- 
.0 20 
.052 
.023 
.032 
• 0 2*». 
. 02t* 

. 0<*6 



-.002 

. oo2 

.001 
-.0C3- 
•.0C8. 

0C2 

• 'A'cl 

-.0C3 

ono 
.out 
. oco 
•. qc2 

-.003 
. OUO 

0C2 
Q CI 



.015 
• 017 
.0^*0 . 
.039 
.02,1 
.Oln 
.016 
L9^ 



r6 

111 ^ 

• 0.26 
.006 

:ni 

• 019 
.019 

.019 



FIT STATISTIC 
AVtRACE RESI0U4L 



1 

-2.75 
. 006 



SUMMARY FIT STATISTICS F.CR APILITY LEVELS 

ABILITY LEVEL CMID-POINTSI 



2 

-2.25 
.00 5 



3 

-1.75 
. 001 



-1.25 
. 000 



5 

.75 

.00 1 



6 

-.25 

.006 



7 

.25 
.002 



8 

.75 
-.001 



AVERAGE- ABSOLUTc RESIDUAL .^.•^J 99 .0 56 .03^ k .022 .016 .017 .012 .Ol<» 

RCOr ^EAS SQUA^^ RESIDUAL .129 . 075 .U<#t* .02fl . 0 2u .025 .017 . 019 



1.55 
-.00 8 



10 

1.75 

-•uoc 



.<^^ .0 25 
.035 



.0 21 



11 

2.25 
-.00^ 



• 039 
.0 60 



I! 



.75 . 
• 000 



»039 



.059 



I 
I 



OVERALL AVERAGES AVERAGE RESIOUaL= 



.GOO AVERAGE ABSOLUTE RESI0OAL= 



.033 ROOT MEAN SQUARE RESICUaL= 



>0(«<» 




233 



234 



stanoarcizec residuals- 3 paraheter MCCEl 




1 2 3 ^' s e 7 

MIO-POINT -3.753 -2. 2 50 -1 .750 -1 . 250 -. 750 -. 250 .2 50 

£Xir?S€eS • 2U 50 11^ 1^^ 318 UUO 509 

^^1^ . 72 3 -.696 .262 -.226 .015 .«99 -.t*28 

2 M2 1 . 197 -. 33 2 -.123 -.152 -. 235 . 2 

3 2!u3t -llesa -.'♦5<# -.325 1.199 -1 .316 .9t.« 
2 -1.0*42 1 . 230 -. 222 .66 e . 797 .3 55 -.912 

1«9 .310 i?66 .ua5 -.218 -.25 9 .171 



- .696 




26 2 




.226 


1 .197 




33 2 




.123 


1.650 




'♦5<# 




.325 


1. 23 0 




22 2 




.66e 


.310 
- . (61 


1. 


m 


1 


.1*85 
. 71 3 


- .5U 1 




8UC 




.601 




1- 


C7& 
633 




.096 
.323 
.097 


1.22 8 


ll 


C12 





LEVEL 
8 




10 


11 


12 


. 750 


1.250 


1 . 750 


2.250 


2.750 


368 


2t.8 


90 


32 


11 


.3^9 
.079 

.»eu 

.207 
.527 
-.225 
-1.0^7 
2.C6U 
-1.713 
.917 


-1.638 
-.332 
.3U6 
-.582 
-•8t«0 
-.690 
-.827 
-.320 
.066 
1.192 


. 953 - 
.953 
.953 
. 953 
-.593 
-.71if - 
-. lt«l 
2. 162 
• 2 20 
-. 106 


1.2C0 
.569 
• 569 
.569 
.569 
1.0 <(8 
.529 
1.375 
-.5(7 
.569 


. 333 
.333 
.333 
-2.697 
-2.697 
. 59*» 
.^•71 
1.25C 
.591 
.333 



f, -Ul/e -.(61 1.887 1. 71 3 -.791 -.77*. 1 . 2l 6 

7 -. 526 -.5U1 .8UC .601 .698 .7m ^.67^ 

8 2 780 1 .830 1-C-7& .096 -.652 -1 .586 -2.t»6^ 

9 1^75 .025 ^. 633 -1.323 . 260 1.726 -.123 
iJ :. 351 1.22 8 1.C12 .097 .3V0-1.U7O -.587 

11 -1.7U7 .521 ^. 536 1. 36 C .U5C -1.2^6 1.377 .9^5 .953 . 569 .333 

12 -^. 676 -l.<7 6 2.U63 l.l 55 1. 32U -.671 -1.7t.2 .8 80 .9t.5 .953 .569 .333 

13 -L dU5 -.^96 - 225 2.312 -.'♦7U 1.260 -.t.Q5 -.1 66 . 3 )6 . 953 . 569 .333 
ll V -I 812 .036 1 868 -.'79C . 8 71 .3 67 -1 .776 .356 1. 583 .953 .569 . 33? 

15 \-^ lil - 751 .7U3 1.061 -.096 1 .b 6q .195-2.925 -. 332 .953 .569 .333 

16 -18^5-1875 . 215 1 .298 2.0l5 1 . C 2? -1 . 67 1 -2 . 8 29 - 1. 75 5 . 953 ,569 . 333 

17 ^ >^7l -.976 . 56U -.871 -.^♦92 2.227 -2. 172 .3 63 .288 -.t«30 .5^*3 .639 
\l 15^6 .353-1^61-1379 - 832 -.1 ^5 1.122 -. 013 .3 6? .375 -2. 616 -.305 
19 -1:721 'l^li -y.7k% -1.>U5 -1. 038 2.90U .729 -2.658 -1.956 .1 87 .$69 .35| 
^0 -1 050 .JOU .D92 -.5U9 .5<»7 -.097 -.036 .0 73 -1. 608 .?53 .569 .333 

21 -. 67 2 .CD 2 .582 -. 725 -.368 1.102 -.U3 8 51-1.218 

22 351 378 1. 22 1 -.92^ -.665 -.^ 26 .058 .U^t .^5^*5 

23 -. 023 '.fl'^a --381 1 . 137 -. 5 73 1 .576 -.292 .123 -.7,0^ 
IS .6^9 -. 725 . 978 1. 161 -1.5 90 .625 1.Q5U -.3 27 -1.38.9 

25 - ?7 1 - 1. 809 -2. 31 7 -;.5t»l .^38 1 .659 .3^9 -1.6i^ -•3ll 

26 ?!2i*6 2.7U3 .76 0 -I.2C8 .619-2.922 -.93<« 1.887 .103 



. CD2 




582 




. 725 


.378 


i! 


22 1 




.92<* 


- .9U8 




381 


1 


.137 


- . 725 




97 8 


1 


.161 


-1.809 


-2! 


31 7 




.5i»l 


2. 7U3 




76 0 


:i 


.208 


- 1 . JB 3 




158 




.965 


. . 793 




522 




.<* 71 


1. f.76 


-1! 


at4*4 




. 15 0 


.58U 


1. 


8t4 7 




.t«63 



.953 


.569 


.333 




.569 


.333 
.t«32 


1. 023 


1.387 


-.337 


1. im 


.6«*5 


-.536 


-.0 ill 

1. 582 


.392 


1. 712 
1. 255 


• 786 


.356 


. <«0^ 


1. 996 


-.631 


-1.0<i« 


-.85n 


-.199 


-.527 


1. 513 


. 8 80 


.333 



27 -.7U? -1.JB 3 . 158 -.965 1.1 76 .5gi . 700 -1 .627 .739 

28 . 678 .. 793 . 522 .^71 .;»t»l 7.l|2 -.516 -223-1.^6^ 

29 '.b^^ l.f.76 -l.a'4^4 -.150 .916 1.1^9 -1.673 . 8 67 -. 122 
I.C2 .58U 1.8t47 -.t.63 -1.0 56 -.171 .t.33 -.t»92 .7U1 

11 1 . 16 8 2 .15 9 . 07 2 -.b^b .17C -.^28 .'♦56 - . 5 <4C -.3^0 1. 366 -1. 2C5 ^.333 

l\ 1 I «22 .7^1 -.39t» I.381 -. 331 .189 -.8 09 .087 1. 173 -1. 2C8 -2.697 

?7 i s;? 92 2 1 u3S -.133 -.360 -. 052 .5 3U-l.t»36 -.301 1.0 50. . 936 

li . 128 (3 9 - 251 -. 889 .1 63 .V23 -. 25; -.639 .533 .953 M .353 

"7? ulstk 3.-715 .61^* -.8t«8 -.167 -2 .286 -2.t«.)3 1.217 1.997 2.093 1. 683 .81<* 

U E55 - 933 6^9 1 . 383 2.6§7 .900 -.858 -.8 96 - 2. 270 -.Sl^^ 1,313 -l.US 

37 . 135 1 .(07 -. 262 1. 102 -.789 .305 -.^0 .687 --S^C -. 258 -.3|l 1.0|| 

18 -1 .'77-8 ^t. 80 5 -1. 351 . ^♦52 1.033 -.lt»7 -1.3 73 .0 95 .306 .953 .569 . 333 

I? i'Ml -ilU r^iU '^Itl "Mi 'im 

23j. . 



<«1 

us 

-^2 



91 
52 
53 
5** 
55 
56 
57 

5a 



098 

5A0 
. <;37 

. 759 
. 560 

► tl2 

> 02U 
. 066 
. 7f>5 



-I 



1. ?5 6 
'2. 419 
-1.79'5 
CJ9 
-.115 
1 .993 
. CI 3 
. 762 



o: 3 

572 
C31 
, u 
539 
, 620 
,198 
, 6 
, 672 



532 
352 
662 
692 

■ 51 3 
76 9 

, <49 2 

. 665 



• £U 3 

1 • 5 
. -33 
-2. 137 
- .78 2 

- . ;62 

-.7 11 
-. •6^5 



-1, 



25. 
- .0 73 
'Z.^71 
•1.156 
. 8 50 
•i. . 196 

.960 



-.50 ; 
.W52 
•1.267 
•1 ♦<*33 
1. 3 2^1 
-1.65<4 
-.<49C 
. 62 3 



- .596 

3.120 
-♦356 

- . 663 

.056 
.137 



-.790 
1.312 

.50 3 
1 .496 

.037 
-.560 

.5<4 3 
-. 996 



-.159 
1.5*47 

-.9<*6 
-.059 
*<«3<« 
-.0 97 



-.^♦5 7 
1.92^ - 

-.371 

-.990 - 
•2. 005 
.536 
■'1.76i* 

1.5 7<4 



.057 
3.625 
f73 
1. 736 
Z.Ull 

. 967 
-•767 
-. 660 



-. 951 
•2. 223 

-i.96<4 

■.Win 

.569 



•MS 

.333 
•1.961 
•2.93f 
.556 
. <49 3 
.121 
. 305 
\.3 j3 



25 3 

32 3 
377 
9<«3 
, i.d7 
, 751 
. 11^ 



.063 
.017 
.<437 
-1.720 
2.110 
-.915 
-.6^9 



.096 
.3 76 
-.711 
•1.0 26 
-.260 
.979 



'Mi 

.002 
2. ICQ 

.369 
-.96i» 
- .70A 
1 .239 



-.162 
-•567 
-.i*73 
.151 

. 54^3 
.26 2 
1.395 
"•1.33<4 



• 2 S6 
,5 66 
3/ 
^72 
.616 
.583 
.<4 22 
•1 .UU6 



'1.2<*9 
1.232 
.572 
•2.939 
• 1.161 
-.2u6 
•1.3<46 
,133 



• '♦26 
.1^7 
.556 
•1.6i*7 
-.911 
-.9^2 
•2,151 
. 610 



1. 741 

17 



1.351 -1.5U5 

. <»93 
,333 
.563 
.35U 
.36*4 
.333 



.001 

6 n 

•1.079 
. 616 

-.91*4 

. 639 



ANfiLVSiS 0=- SrAv^DAR0l7ED RCSlOUALS 
(ABSOLLTE VALUES! 

- ^ CUMULATIVE 

f Ef^CENT 



INlE^iVAL NJiBER^ PERCENT 



C TO 1 
1 TO 2 
5 TO 3^ 
BEYONO 3 



^♦57 
1 61 

52 



65 . 66 
26. u 1 
7. ^.7 
. 66 



65.66 
91 .67 
99.14 
103. DO 



SUMMARY OF FIT JTaTISTIcS FOR ITEmS 

,rs!sssE "ifsSiuJ?--" vAm.i^u^ ^"-jiijjjir^ %Jii!i!2^;iisvgK. 

3 -29} -Hi \*\i7 . 036 -fel^ 

'^^^ • i:?c .2/0 . .^06 



:63s. i>>2 i;||3 ':iy^ .gij 

9 .111 .032 



A 



" ••iff ■ '-.Ml 



15 -.10? " 922 1 coi -.2f,5 1.790 



23u 



Ill 




19 




20 


2L 


_ » » 7 


22 


• 1? - 

• 3 9 


c *♦ 


1 ^ t 

-•'♦69 


cS 


CO 


Cs. 1 > 
• D I 




• • 1 J O 






9Q 








-o-U^ — 
3 1 


— •-t-i-i — 

? < 7 


0 c 


• Q ) ^ 


33 


• 1 3 3 




• 2 5 9 


23 




Ob 


n 1* ? 


17 

or 




1 A 

<S Q 


• f* - q 


ta 


« 9 7 ^ 
— • C ' ■ 


^ i> 
**« 


• 13 7 
• X ' f 


H I 


.C83 


^ 2 


T ft A 

• 3 fJ 


U J 




HH 


• 1 • too 


1^ C 


• 1 n 1 7 




- • 0 S 'J 
• 01 V 


!♦/ 




_ n * Ji 


'♦9 


_ 1 > » 

1 




• 1 3 0 


5 1 
15 


• U U 1 
r* 4 Q 

• t I ^ 

• 15 0 


5V 


-•652 


55 


-•319 


56 


• 02?; 
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